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(57) A program ancillary data processing apparatus 
is provided which includes a program ancillary data ex- 
traction unit for extracting program ancillary data perti- 
nent to commodities or services appearing in each 
scene, from picture program data, and a display control 
unit for displaying, on a display picture, the information 



relevant to the commodities or services appearing in 
each scene, and the sum total of the number of times of 
scene-based access events by the audience to a data- 
base supervising the detailed information relevant to the 
commodities or services, based on the preset layout in- 
formation. 
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Description 

Technical Field 

[0001] This Invention relates to a program ancillary 
data processing device in a digital picture program re- 
ceiving reproducing system, a server device, a program 
information displaying method and a recording medium 



Background Art 

[0002] In certain programs aimed to sell goods (so- 
called television shopping), the viewer's will to purchase 
goods can be communicated to a server through a tel- 
ephone network or the like up link. In this sort of the pro- 
gram, services can be rendered by embedding the an- 
cillary information necessary in purchasing the com- 
modities, inclusive of the telephone number for connec- 
tion to the server in making an offer for purchase, in a 
gap of the analog broadcast waves. However, with the 
currently operating broadcast system, it is not possible 
to supply a sufficient amount of the information to the 
viewer because of constraint in e.g., the signaling sys- 
tem. However, in the digital broadcast system, as used 
in the next-generation broadcast system, in which an 
appreciably large quantity of the information can be 
transmitted, there is a good prospect for realization of 
variegated services employing the program ancillary da- 
ta. For example, there is a good prospect for realization 
not only of an interpolating service of checking for a sing- 
er's profile as the song program on air is viewed, but 
also of a bidirectional service of placing an order for 
commodities presented on a display picture as a picture 
program on air is viewed. 

[0003] However, even if the amount of the information 
acquired Is increased, but the access to the required in- 
formation is not facilitated, It is not possible to render 
attractive services for the viewer. 

Disclosure of the Invention 

[0004] In view of the above problems, the present in- 
vention is aimed to provide a program ancillary data 
processing device, a server device, a program informa- 
tion displaying method and a recording medium in a dig- 
ital picture program receiving and reproducing system, 
in which the access by the viewer to the information is 
facilitated. For solving the above problem, the following 
means are proposed. Meanwhile, a receiving reproduc- 
ing system for digital picture programs may be con- 
structed which gives rise to variegated user friendliness 
depending on the combinations used. Of course, an on- 
line shopping system can be constructed by intercon- 
nection to the external database over a network. Mean- 
while, the undermentioned respective means do not 
have to be mounted in the territory of the same nation 
at the time of the network system construction. As for 
recording mediums, by optionally combining the under- 



mentioned means, such a recording medium may be re- 
alized which gives rise to variegated user friendliness 
depending on the combinations used. 
[0005] The functional units, forming the undermen- 
5 tioned means, may be implemented as hardware or as 
software. 

[0006] As first means, the present invention proposes 
a program ancillary data processing apparatus including 
a program ancillary data extraction unit for extracting 
10 program ancillary data relevant to commodities or serv- 
ices appearing in each scene, from picture program da- 
ta, and a display control unit for displaying, on a display 
picture, the information relevant to the commodities or 
services appearing in each scene, and the sum total of 
the number of times of scene-based access events by 
the audience to a database, which supervises the de- 
tailed information relevant to the commodities or serv- 
ices, based on the preset layout information. 
[0007] With use of this program ancillary data 
20 processing device, in which the degree of interest the 
audience at large feels in commodities or services ap- 
pearing in a picture program can be checked on the 
scene basis, with the number of times of browsing of the 
detailed information and the number of offers for pur- 
25 chase, as indices, it can be readily confirmed whether 
the commodities or services contained in a scene are 
popular ones. In particular, the device can be used ef- 
fectively for browsing program ancillary data for many 
scenes. 

30 [0008] The 'program ancillary data' may encompass 
not only data received over a transmission path, but also 
data reproduced from a recording medium. Meanwhile, 
the recording medium may be the picture program re- 
ceived, reproduced and recorded by the user, in addition 
35 to the recording medium offered to the market as It has 
the picture program data pre-recorded thereon. If the 
picture program data has been received over a trans- 
mission route, the receiving device for picture program 
data may be provided in the same casing as that for a 
40 program ancillary data processing device, or may be 
provided as a separate device in a separate housing. In 
case the picture program data is available over the 
transmission route, the latest value of the number of 
times of access events can be demonstrated substan- 
45 tially in real-time. 

[0009] Asforthetransmission route, it does notmatter 
whether the transmission route is a wired path, such as 
copper wire or optical fiber, or a radio route, such as 
short wave, ultra-short wave, micro-wave or infrared 
so light. As for the transmission system, it does not matter 
whether the system is a broadcast system or an inter- 
active transmission (distribution). 
[001 0] If the picture program data is reproduced from 
a recording medium, the reproducing system for repre- 
ss ducing picture program data may be provided in the 
same casing as that for the program ancillary data 
processing device, or provided as a separate device 
provided in a separate casing. When the picture pro- 
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gram data is reproduced from the recording medium, 
there is a time difference between the time of recording 
the number of times of access events displayed and the 
time of reproducing the number of times of access 
events displayed, so that the latest value cannot be fur- 5 
nished in real-time. However, the number of times of ac- 
cess events displayed can be used as a more or less 
reliable measure. 

[0011] Meanwhile, if the correct information is to be 
demonstrated in case of using the system of reproduc- 
ing the information from a recording medium, it may be 
contemplated to access the database supervising the 
picture program data in question, over a separate com- 
munication network, to receive the latest number of 
times of access events. 

[0012] As the recording medium, a CD-ROM, a medi- 
um of the magnetic recording system, such as magnetic 
tape, floppy disc, hard disc or magnetic card, an optical 
recording type medium, such as write-once CD, DVD or 
an optical card, a magneto-optical recording type medi- 
um, such as DVD, MD or MO, or a semiconductor mem- 
ory, such as non-volatile memory or RAM, may be used. 
[0013] The 'scene' may be a spot, shot, a frame or a 
field. The 'information relevant to commodities or serv- 
ices' may, for example, be a brand name, a category 
name of a commodity, name of a commodity or service, 
name of a producer, name of a supplier of commodities 
or services, contact site, such as URL, an e-mail number 
or a telephone number, or charges. The information 
may, of course, be a combination of part of these infor- 
mation, without encompassing all of the above informa- 
tion. 

[0014] The 'detailed information relevant to the com- 
modities or services' may include, for example, the 
name of the furnishing business operator, a selling busi- 
ness operator, a possible site of purchase or furnish- 
ment, time period, reserve, furnishing time period, terms 
of delivery, remuneration, conditions of payment, com- 
modities, body color sorts or service image information, 
and detailed legends. The information may, of course, 
be a combination of part of these information, without 
encompassing all of the above information. 
[0015] Meanwhile, if the GPS information is included 
in the above information, it may be associated with a 
map database so as to be used in grasping, at the out- 
set, the site of possible purchase on a display picture. 
Of course, one can arrive at a destination, without get- 
ting lost, by delivering the position information to a car- 
laden GPS receiver via e.g., a recording medium. 
[0016] The superintendent of the 'database', which 
supervises these detailed information, may be a produc- 
er or a distributor of a recording medium or a business 
undertaker, operating as an intermediary for purchasing 
commodities or services, in addition to the producer, 
broadcaster or distributor of picture programs. The 'da- 
tabase' is not simply the information supplier, but may 
also be a supplier for an on-line shopping function. At 
any rate, it is sufficient if the 'database* has in store the 



information which allows to grasp the correlation of the 
information in store with respect to the commodities or 
services appearing as an imaged object in a picture pro- 
gram by computerized data processing. 
[0017] The 'number of times of access events by the 
audience' may be the information relevant to the sum- 
mation of the number of times of access events which 
failed to led to purchase and the number of times of ac- 
cess events which led to purchase, or the information 
relevant to one of these two numbers of times of access 
events. 

[0018] The 'results of the summation of scene-based 
access events' encompass not only a case in which the 
number of times of access events is counted in a counter 
designed for counting the number of times of access 
events, based on the information received by the pro- 
gram ancillary data processing device, using the 
number of times of access events to the individual com- 
modities or services, but also a case in which the 
number of times of access events is already counted in 
a sending or recording side of the program ancillary data 
and received as program ancillary data. Which of these 
systems is adopted depends on the transmission format 
or on the recording format. The information may also be 
acquired separately over a communication network. 
[0019] The 'layout information' may be included as 
part of the picture program data, for example, as the tag 
information of the picture description language, or may 
be saved apart from the picture program data by the pro- 
gram ancillary data producing device, for example, as 
the template jnformation. Meanwhile, this 'layout infor- 
mation' may or may not include the information for cor- 
relation with an index picture for a corresponding scene. 
If the correlation information is included, an index picture 
for a corresponding scene may be demonstrated as 
necessary. 

[0020] The terminal operating as this 'program ancil- 
lary data processing device* may include computer ter- 
minals, such as notebook or desktop personal comput- 
ers, PDA, portable telephone sets, game machines, set 
top boxes, PC card, domestic electrical utensils, VTR, 
television receivers, car navigation systems, wrist 
watches having the information processing functions, 
distribution terminals for music or games, financial ter- 
minals (ATM) etc, in addition to dedicated terminals for 
processing the program ancillary data. The same ap- 
plies for the program ancillary data processing devices 
that follow. 

[0021] As second means, the present invention pro- 
poses a program ancillary data processing apparatus 
including a program ancillary data extraction unit for ex- 
tracting program ancillary data relevant to commodities 
or services appearing in each scene, from picture pro- 
gram data, and a display control unit for displaying the 
information relevant to the commodities or services ap- 
pearing in each scene, and the sum total of the number 
of times of commodity-based or service-based access 
events by the audience to a database, supervising the 
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detailed information relevant to the commodities or serv- 
ices, on a display picture, based on the preset layout 
information. 

[0022] This second means is similar to the first means 
except that the results of summation of the number of 
times of accessing events by the audience on the com- 
modity or service basis are demonstrated on the display 
control unit. Thus, with use of this program ancillary data 
processing device, the degree of interest of the audi- 
ence at large feels in commodities or services appearing 
in a picture program can be checked on the scene basis 
with the number of times of browsing of the detailed in- 
formation and the number of offers for purchase as in- 
dices, so that it can be readily confirmed whether the 
commodities or services contained in a scene are pop- 
ular ones. For example, this means is convenient in 
case limitations are imposed on the possible number of 
offers for purchase or of items that can be purchased 
[0023] The 'results of summation on the commodity 
or service basis' are usually calculated on the sending 
side or recording side of picture program data. These 
information not only may be read out from the program 
ancillary data but also may be acquired over a separate 
communication network. Meanwhile, the explanation on 
other terms common to the above-described means is 
the same as that for these means. 
[0024] In the above explanation, the function for im- 
plementing the first means and the function for imple- 
menting the second means are mounted separately 
These functions may, however, be loaded on one de- 
vice. That is, the function of displaying the results of 
summation on the scene basis and that of displaying the 
results of summation on the commodity or service basis 
may be loaded on one and the same device. 
[0025] As third means, the present invention propos- 
es a program ancillary data processing device of the first 
or second means wherein the results of summation of 
the number of times of scene-based access events the 
summation of the number of times of commodity-based 
access events and the summation of the service-based 
access events from the audience to the database, which 
supervises the detailed information on the commodities 
or services, are acquired as program ancillary data of 
the picture program data. 

[0026] This program ancillary data processing device 
in which the targeted information on the number of times 
of access events is acquired from the program ancillary 
data received or reproduced with the picture program 
is convenient as a device not carrying communication 
means. It should be noted that, if the device includes 
communication means, the time until display of the rel- 
evant information can be shortened by acquiring the in- 
formation on the number of times of access events from 
the so received or reproduced program ancillary data 
[0027] Meanwhile, the explanation on other terms 
common to the above-described means is the same as 
that for these means. 

[0028] As fourth means, the present invention propos- 



es a program ancillary data processing device of the first 
or second means wherein the results of the scene- 
based summation, commodity-based summation or the 
service-based summation of the numbers of times of ac- 
s cess events by the audience to the database supervis- 
ing the detailed information pertinent to the commodities 
or services are acquired, apart from the picture program 
data, from the database which is connected to the de- 
vice over a communication network and which supervis- 
ee es the detailed information on the commodities or serv- 
ices. 

[0029] This program ancillary data processing device, 
in which the information on the number of times of ac- 
cess events can be acquired over a communication net- 
's work, apart from the picture program data, is able to fur- 
nish the latest number of times of the access events at 
all times to the audience, not only for the case in which 
the contents of the program ancillary data are confirmed 
simultaneously with the progress in the display of the 
20 pic ture program but also forthe case in which the picture 
program data transiently stored is reproduced or in 
which the picture program data distributed as the re- 
cording medium is reproduced. 
[0030] It does not matter whether the 'communication 
25 network* is a wired path or a radio path. Meanwhile, the 
explanation on other terms common to the above-de- 
scribed means is the same as that for these means. 
[0031] As fifth means, the present invention proposes 
a program ancillary data processing device including a 
30 program ancillary data extraction unit for extracting 
from picture program data, program ancillary data rele- 
vant to commodities or services appearing in each 
scene, and a display control unit for displaying the infor- 
mation relevant to the commodities or services appear- 
as mg in each scene, on a display picture, along with an 
index picture of a corresponding scene, based on the 
preset layout information. 

[0032] With use of this program ancillary data 
processing device, the audience is able to confirm the 
^o contents of each scene forming the picture program in 
particular the contents of the commodities or services 
provided by the program ancillary data, as he or she 
compares them with the display on the index picture. 
[0033] Meanwhile, if the program ancillary data is dis- 
<* played at the same time as the picture program is dis- 
played, similar objectives may be achieved from the dis- 
play of the picture program. However, since the index 
picture is usually displayed as a still picture, in distinc- 
tion from the picture program, the information required 
so may advantageously be confirmed at a slow pace. 
[0034] Of course, if only the contents of the program 
ancillary data are to be displayed, apart from the display 
of the picture program, the contents of a scene may be 
confirmed from the index picture, even if the user relies 
55 on his or her memory, so that the user is able to view 
only the picture program itself, while he or she may con- 
firm the contents of the program ancillary data at a slow 
pace after the end of the picture program. This means 



4 



7 EP 1 335 592 A1 8 



releasing the audience from the viewing time constraint. 
[0035] As the 'index picture', a typical frame picture of 
a corresponding scene in general is used. The index pic- 
ture may, for example, be a leading frame of a relevant 
scene, a f rame wh ich allows for easy recognition of com- s 
modities or services, such as a frame decided on by a 
producer or by the audience, or a trailing frame of the 
relevant scene. Meanwhile, the index picture is desira- 
bly a picture taken out or generated from the picture pro- 
gram, for confirming the contents of the scene, and may 
also be a computer picture or an icon that allows for dis- 
crimination of effect sounds (theme songs, inserted 
songs and BGM). In the case of the effect sounds, a 
frame picture of a scene employing the effect sounds or 
an image picture of the music sound may similarly be 
used. 

[0036] Although the index picture is a still picture, as 
a principle, the index picture may also be provided with 
a moving picture reproducing function in order that the 
audience is able to be readily reminded of the contents 
of the relevant scene. The moving picture reproducing 
function does not have to be the full picture size repro- 
ducing function, such that the reproducing function of a 
simplified picture having a reduced picture size or hav- 
ing the number of pixels reduced on decimation suffices. 
The reproducing speed may be such as permits variable 
speed reproduction. The index picture may be provided 
with the music sound reproducing function. This repro- 
ducing function does not have to be music sound repro- 
duction throughout the entire scene such that reproduc- 
tion of the music sound of the representative portion 
may be sufficient. The reproduction may also be varia- 
ble speed reproduction. 

[0037] If the display function of the index picture in the 
present means and the function of displaying the 
number of times of access events, explained in connec- 
tion with the above-described means, are combined to- 
gether, compound effects may be derived. Meanwhile, 
the explanation on other terms common to the above- 
described means is the same as that for these means. 
[0038] As the sixth means, the present invention pro- 
vides the fifth means wherein the index picture is given 
as program ancillary data for the picture program data. 
[0039] With use of this program ancillary data 
processing device, in which the index picture does not 
have to be prepared in the device itself, the processing 
resources necessary for carrying out the processing can 
be allocated to other processing operations. In such 
case, there are provided on the audience side an index 
picture extraction unit and a storage unit for storing the 
extracted index picture. 

[0040] In particular, if the considerable processing ca- 
pability is required in the preparation of an index picture, 
or if significant instabilities in the processing capability 
are predicted to be produced in a receiving terminal or 
in a reproducing terminal, it is desirable to prepare an 
index picture at the outset and to have the so prepared 
index picture included in the program ancillary data. 



Meanwhile, the explanation on other terms common to 
the above-described means is the same as that for 
these means. 

[0041] As seventh means, the present invention pro- 
poses the fifth means wherein the index picture is cre- 
ated on the audience side from picture data of picture 
program data. 

[0042] With use of this program ancillary data 
processing device, in which the index picture is pre- 
pared within the device itself, it is only sufficient if the 
index picture as required in case of necessity is pre- 
pared, while it is possible to evade the risk of an abun- 
dant recording area being consumed by the index pic- 
ture of the major portion of program ancillary data that 
does not have to be confirmed. Since the index picture 
does not have to be included in the picture program data 
in such case, the amount of data transmitted or recorded 
can be reduced, thus reducing the amount of data trans- 
mitted or recorded. In this case, an index picture former 
is provided on the audience side. 
[0043] It is noted that the program ancillary data 
processing device may be adapted to prepare an index 
picture only when the program ancillary data includes 
the information on the commodities or services, in which 
case the decision unit for discriminating the presence/ 
absence of the information in question is provided on 
the audience side and the permission to prepare an in- 
dex picture is given the index picture forming unit only 
when there exists the information on the commodities 
or services. Alternatively, the index picture may be pre- 
pared irrespective of the presence or absence of the rel- 
evant information. 

[0044] For specifying a picture used as an index pic- 
ture, the information set at the outset in the device itself 
may be used, in which case the information is set on the 
recording medium having stored the position informa- 
tion. Alternatively, the picture may be specified by pro- 
gram ancillary data, in which case there is provided a 
position specifying information readout unit. 
[0045] Meanwhile, the explanation on other terms 
common to the above-described means is the same as 
that for these means. 

[0046] As eighth means, the present invention pro- 
poses the seventh means where the index picture has 
been produced from a scene optionally selected by the 
viewer. 

[0047] With use of this program ancillary data 
processing device, not the index picture in which com- 
modities or services relevant to program ancillary data 
appear in an averaged out manner, but a picture which 
permits facilitated recognition of the commodities or 
services the audience intends to confirm can be select- 
ed as an index picture, so that the picture selected may 
be convenient for the viewer. 

[0048] It should be noted that an input accepting unit 
for a scene position as selected by the viewer, some- 
times including the position information for the relevant 
scene, a recording unit for recording the input position 
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information and an index picture former are provided on 
the viewer side. 

[0049] Meanwhile, the explanation on other terms 
common to the above-described means is the same as 
that for these means. 

[0050] As ninth means, the present invention propos- 
es a program ancillary data processing device including 
a program ancillary data extraction unit for extracting 
program ancillary data relevant to commodities or serv- 
ices appearing in each scene, from picture program da- 
ta, and a display control unit for displaying the informa- 
tion relevant to plural scenes, as extracted, based on 
the layout information for list type display. 
[0051] With use of this program ancillary data 
processing device, in which access to the information 
relevant to plural scenes can be made on one picture, 
it is possible to evade the laborious operation of con- 
firming the contents from one scene to another. Moreo- 
ver, since the information of plural scenes can be com- 
pared to one another on the same picture, the targeted 
information can be accessed more readily. 
[0052] The file for list type display, such as a list file, 
may or may not be contained at the outset in the 'pro- 
gram ancillary data'. In the former case, It is sufficient if 
the file designed for list type display is read out and dis- 
played, whereas, in the latter case, it is sufficient if the 
file for list type display is formed on the basis of the in- 
formation for each scene. A list file former is provided 
on the viewer side. 

[0053] It should be noted that the range of plural 
scenes displayed in a list form may be afforded as a part 
of the program ancillary data, or determined under the 
commands of the program ancillary data processing de- 
vice, that is under the commands by the viewer. 
[0054] In the latter case, an automatic list range deci- 
sion unit may be provided with the function of setting 
plural scenes, lying ahead and at back of a scene the 
viewer has requested to be displayed in a list form, as 
a display range. Alternatively, the automatic list range 
decision unit may be provided with plural scenes previ- 
ous to the scene the viewer requested to be displayed 
in a list form (past scenes already displayed) or plural 
scenes subsequent to the scene the viewer requested 
to be displayed in a list form (scenes not yet displayed). 
Meanwhile, the automatic list range decision unit may 
be provided with a function which, when the viewer has 
specifically indicated the range (the start position and 
the end position), sets the so specified range as the dis- 
play range. 

[0055] In general, the range of plural scenes may be 
the whole program, encompassing the entire scenes, or 
several scenes, such as two or three scenes. This range 
may be given as program ancillary data or automatically 
determined under the command given by the program 
ancillary data processing device. These ranges may or 
may not be overlapped with one another. 
[0056] On the other hand, the 'layout information' may 
be included as part of the picture program data, or saved 



in the program ancillary data processing device inde- 
pendently of the picture program data. Meanwhile, this 
'layout information' may or may not include the informa- 
tion of correlating the relevant scene with its index pic- 
s ture. 

[0057] Meanwhile, the explanation on other terms 
common to the above-described means, such as 'pro- 
gram ancillary data processing device', is the same as 
that for these means. 
10 [0058] As tenth means, the present invention propos- 
es the ninth means wherein the display control unit has 
the function of displaying the information relevant to 
commodities orservices appearing in each scene, along 
with an index picture of a corresponding scene, on a dis- 
15 play picture. 

[0059] With use of this program ancillary data 
processing device, the viewer is able to confirm the con- 
tents of the respective scenes making up a picture pro- 
gram, in particular the contents of commodities or serv- 
20 ices given by the program ancillary data, based on the 
display on the index picture. Thus, anyone (from chil- 
dren to aged people) may easily find out the scene of 
interest from the plural scenes displayed in a list form. 
Moreover, ease in use for the viewer is improved appre- 
25 ciably because he or she is able to confirm the contents 
of the commodities or services appearing in the so spec- 
ified scene as he or she compares the contents with 
those shown in the associated scene. Meanwhile, the 
advantage derived from the use of the index picture is 
30 the same as that in the above means. The site in which 
to create an index picture is also as explained in con- 
nection with the above means. 
[0060] It is noted that the explanation on other terms 
common to the above-described means is the same as 
35 that for these means. 

[0061] As the eleventh means, the present invention 
proposes the ninth means wherein scenes of the picture 
program to be processed are extracted in groups com- 
posed of a preset number of scenes, in the order in 
40 which they are displayed, based on the layout informa- 
tion, for presentation as plural scenes for the viewer in 
a list form. 

[0062] With use of this program ancillary data 
processing device, the scenes displayed in the list form 
« are necessarily contiguous to one another, so that any- 
one (from children to aged people) may easily access 
the target scene. Meanwhile, the 'preset number 1 may 
be included in the program ancillary data, or may be set 
or changed on the viewer side. In the latter case, a re- 
50 cording unit is provided on the viewer side for recording 
the number of scenes usable at the time of display in 
the list form. 

[0063] The expression 'in groups composed of a pre- 
set number of scenes, in the order in which they are dis- 
55 played' does not mean that the presentation of partially 
common scenes between a scene included in given re- 
sults of extraction and a scene included in the next or 
previous results of extraction is to be excluded. 
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[0064] It is noted that the explanation on other terms 
common to the above-described means is the same as 
that for these means. 

[0065] As twelfth means, the present invention pro- 
vides the ninth means wherein scenes of the picture pro- s 
gram to be processed are optionally commanded for se- 
lection by the viewer and extracted in groups composed 
of a preset number of scenes, based on the layout in- 
formation, for presentation as plural scenes for the view- 
er in a list form. 

[0066] With use of the program ancillary data 
processing device, only the scenes are displayed which 
have been selected by the viewer with an intention of 
confirming them. This enables the viewer to access the 
necessary scene easily in a short time. Meanwhile, if the 
volume of the information correlated with the respective 
scenes is increased, ease in accessing the scene de- 
sired to be confirmed may be lost, even although con- 
venience for the user is increased. Thus, with use of the 
device, it is possible to render services convenient to 
use for anyone (from children to the aged). In the 
present device, a recording unit for recording the 
number of scenes at the time of display in a list form is 
provided on the viewer side. 

[0067] It is noted that the explanation on other terms 
common to the above-described means is the same as 
that for these means. 

[0068] Meanwhile, if the ninth to twelfth means are 
combined with the function of displaying the number of 
times of access events, there may be provided a confir- 
mation picture of the program ancillary data which is 
more convenient to use for the viewer. If, in displaying 
plural scenes in a list form, an index picture is afforded 
to each scene, and the number of times of scene-based 
access events or that of the commodity- or service- 
based access events is displayed, the scene targeted 
or the scene in which the detailed information is desired 
to be confirmed can be determined easily. 
[0069] In each of the first to twelfth means, explana- 
tion has been made on the premises that the program 
ancillary data pertinent to the commodities or services 
appearing in each scene are supplied through picture 
program data. Alternatively, these data may also be ac- 
quired from an external database over a communication 
route, as indicated in thirteenth means. 
[0070] As the thirteenth means, the present invention 
proposes a program ancillary data processing appara- 
tus including an extraneous data presence/absence de- 
tection unit for extracting, from picture program data, da- 
ta indicating whether or not the information relevant to 
commodities or services appearing in each scene is 
available through an external database, and for detect- 
ing the presence/absence of the extraneous data based 
on the so extracted data, a downloading unit for down- 
loading, when an access request to extraneous data is 
made by an operating input by the viewer, a list file pro- 
vided from the outset for presenting the information rel- 
evant to plural scenes to the viewer in a list form, from 



an external database, and a display control unit for 
opening the downloaded list file for displaying the infor- 
mation relevant to the plural scenes involved in the list 
file on a display picture in a listform, based on the preset 
layout information. The list file is an information file pro- 
vided from the outset for a relevant picture program, 
[0071] With the present program ancillary data 
processing device, a picture similar to that of the ninth 
means may be provided to the viewer if the information 
relevant to commodities or services appearing in each 
scene is not included in the program ancillary data, but 
there is contained data indicating whether or not the in- 
formation relevant to commodities or services appear- 
ing in each scene is available through an external data- 
base, and the data indicates availability of the informa- 
tion from the external database. That is, access to the 
information relevant to plural scenes can be made in one 
picture to dispense with the labor in performing similar 
operations from scene to scene. Moreover, access to 
the target information can be facilitated because the in- 
formation for plural scenes can be contrasted to one an- 
other on the same picture. 

[0072] Meanwhile, this service configuration has high 
affinity with chargeable services. That is, services can 
be rendered only to users desirous to have more de- 
tailed information furnished to them. In addition, since 
only a small amount of the supplementary information 
to program ancillary data suffices, this service configu- 
ration can co-exist with services at large. 
[0073] The 'external database 1 may be enumerated 
by a producer or a distributor of the recording medium, 
business undertakers acting as intermediary for pur- 
chasing the commodities or services, in addition to pro- 
ducers, broadcasters and distributors of the picture pro- 
gram. This database may also furnish the on-line shop- 
ping function, without being limited to simply furnishing 
the information relevant to commodities or services. 
[0074] The 'data indicating availability' may, for exam- 
ple, be data represented by a flag or plural bits for spec- 
ified addresses. The 'list file' indicates a set of program 
ancillary data for each of the plural scenes belonging to 
the scene range in question. 

[0075] Meanwhile, the plural scenes making up the 
'list file' may be provided at the outset by the database 
superintendent or may be decided on responsive to the 
viewer's selection. In this case, the range selected by 
the viewer is notified to the database over a communi- 
cation route. These scenes may also be those extracted 
at a rate of a preset number in the order in which these 
scenes appear in the picture program. 
[0076] The 'preset layout information' may be sup- 
plied from the database along with the list file, or saved 
at the outset in the program ancillary data processing 
device. The layout information saved in the program an- 
cillary data processing device may be previously sup- 
plied from the database, or saved as from the time of 
shipment of the program ancillary data processing de- 
vice. 
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[0077] The layout information may or may not include 
the information correlating the information with the index 
picture for the corresponding scene. The 'correlating in- 
formation' may be the recording addresses, file numbers 
and so forth. 

[0078] The terminal operating as this 'program ancil- 
lary data processing device' may include computer ter- 
minals, such as notebook or desktop personal comput- 
ers, portable information terminals (PDAs), portable tel- 
ephone sets, game machines, set top boxes, PC cards 
domestic electrical utensils, VTR, television receivers' 
car navigation systems, wrist watches having the infor- 
mation processing functions, or remote controllers in 
addition to the dedicated devices for processing pro- 
gram ancillary data. 

[0079] Meanwhile, the explanation on other terms 
common to the above-described means is the same as 
that for these means. 

[0080] As the fourteenth means, the present invention 
proposes a program ancillary data processing device in- 
cluding a downloading unit for communication with an- 
other terminal device for downloading the information 
which is relevant to commodities or services appearing 
in each scene, and which is program ancillary data of a 
picture program in question, from a recording medium 
supervised by another terminal device, and a display 
control unit for demonstrating, on a display picture the 
information relevant to commodities or services appear- 
ing in each scene, responsive to an operating input bv 
an operator. 7 
[0081] With use of this program ancillary data 
processing device, program ancillary data received or 
reproduced by another terminal device and stored on a 
recording medium may be separately taken out for re- 
use. For example, program ancillary data received and 
stored in a TV receiver having the storage function may 
be downloaded to confirm its contents. In particular the 
program ancillary data processing device may be used 
for such application in which program ancillary data 
stored in a standstill terminal device is transferred by a 
user to a portable program ancillary data processing de- 
vice for confirming the stored information when the user 
is out, or in which the user makes on-line shopping 
based on the so confirmed information. 
[0082] It is of course possible with a portable program 
ancillary data processing device to directly receive or 
reproduce program ancillary data or to download the 
program ancillary data directly from an external data- 
base. However, in view of the currently available device 
in which the communication speed with radio waves felt 
to be a mainstream in the portable device, is insufficient 
or limitations are imposed on the recording capacity the 
demand from the user is thought to be high. Although 
this technical task will be solved in future, it is meritori- 
ous to selectively capture only data required by the view- 
er, in consideration of the labor involved in reaching the 
target data from among the voluminous data. 
[0083] The terminal operating as this 'program ancil- 



lary data processing device' may include computer ter- 
minals, such as notebook or desktop personal comput- 
ers, portable information terminals (PDAs), portable tel- 
ephone sets, game machines, set top boxes, PC cards 
s domestic electrical utensils, VTR, television receivers' 
remote controllers, car navigation system and wrist 
watches having the information processing functions in 
addition to the dedicated devices for processing pro- 
gram ancillary data. However, if the application in this 
10 means is taken into consideration, a standstill type de- 
vice is thought to be customary. 
[0084] As for the' recording medium', it may not only 
be enclosed in another device, but also mounted exter- 
nally on such other device, provided that the recording 
'5 medium is supervised by such other device. The record- 
ing medium may be a CD-ROM, a medium of the mag- 
netic recording system, such as magnetic tape floppy 
disc, hard disc or magnetic card, an optical recording 
type medium, such as write-once CD, DVD or optical 
20 cards, a magneto-optical recording type medium, such 
as DVD, MD or MO, or a semiconductor memory, such 
as non-volatile memory or RAM. 
[0085] Meanwhile, the'information relevant to com- 
modities or services appearing in each scene' may in- 
25 dude the sum total of the number of scene-based ac- 
cess events from the viewer to the database, which su- 
pervises detailed information concerning the commodi- 
ties or services in question. The 'information relevant to 
commodities or services appearing in each scene' may 
30 also include the sum total of the number of commodity- 
based or scene-based access events from the viewer 
to the database which supervises detailed information 
concerning the commodities or services in question. In 
this case, these information may be acquired from other 
55 terminal devices or directly downloaded from the data- 
base which supervises the information. The 'information 
relevant to commodities or services appearing in each 
scene' may include index pictures associated with re- 
spective scenes. 

40 [0086] The 'program ancillary data processing device' 
for downloading data from another terminal may include 
computer terminals, such as notebook or desktop per- 
sonal computers, portable information terminals 
(PDAs), portable telephone sets, game machines, set 
top boxes, PC cards, domestic electrical utensils VTR 
television receivers, remote controllers, car navigation 
systems and wrist watches having the information 
processing functions, in addition to the dedicated devic- 
es for processing program ancillary data. However, if the 
50 application in this means is taken into consideration a 
standstill type device is thought to be customary Al- 
though it is desirable that the 'program ancillary data 
processing device' has a display device enclosed there- 
in, it is sufficient if the contents are displayed on the dis- 
55 play device even if the display device is not enclosed 
[0087] Meanwhile, the explanation on other terms 
common to the above-described means is the same as 
that for these means. 
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[0088] As the fifteenth means, the present invention 
proposes the fourteenth means wherein the display con- 
trol unit has the function of presenting the information 
relevant to plural scenes to the viewer in a list form 
based on the preset layout information. 
[0089] With use of this program ancillary data 
processing device, in which the information relevant to 
plural scenes can be confirmed at a time on the display 
picture, there is no necessity of performing the operation 
of demonstrating the information of another scene even 
when the information concerning a given scene does not 
include the target information. As a result, the retrieval 
is more effective than in the case of accessing each 
scene separately. Meanwhile, the list-form picture may 
be downloaded from another terminal device, or pro- 
duced in the program ancillary data processing device. 
[0090] The number of times of scene-based access 
events or that of commodity- or service-based access 
events may be demonstrated for each of the scenes dis- 
played in a list form. Similarly, the index picture may be 
demonstrated in association with each scene. In this 
case, these information can be directly downloaded 
from an external database. 

[0091] Meanwhile, the function of the fifteenth means 
may be provided by itself, instead of as a combination 
with the fourteenth means. 

[0092] It is noted that the explanation on other terms 
common to the above-described means is the same as 
that for these means. 

[0093] As the sixteenth means, the present invention 
provides a program ancillary data processing apparatus 
for displaying the information relevant to commodities 
or services appearing in each scene, on a display pic- 
ture, based on program ancillary data extracted from 
picture program data, in which the apparatus includes 
a display control unit for displaying an index picture of 
a scene, in an imaged object of which are included com- 
modities or services selected with a will for purchasing, 
in a correlated fashion in a basket picture in which the 
contents of the commodities or services selected can 
be confirmed prior to final decision of transaction. 
[0094] With use of this program ancillary data 
processing device, when the commodities or services 
scheduled to be purchased on a basket display picture 
are re-confirmed, it is possible to make a re-check of the 
commodities or services on an index picture. As a result, 
the labor of exiting the index picture to move to the other 
picture only for the purpose of confirming the contents 
of the commodities or services may be dispensed with. 
In particular, when the commodities being confirmed on 
the basket picture affect several scenes, it is unneces- 
sary to return to each scene. This gives a confirmation 
picture convenient to use readily by anyone. 
[0095] Meanwhile, the number of times of scene- 
based access events or the number of times of com- 
modity- or service-based access events can be dis- 
played on the basket picture. If this function is combined 
with the above function, the commodities or services 



can be checked for popularity among other users with- 
out the necessity of moving from one picture to another 
picture. 

[0096] Although the case of displaying the index pic- 

5 ture on the basket picture has been explained in con- 
nection with the present means, it may be contemplated 
to display only the number of times of scene-based ac- 
cess events or only the number of times of commodity- 
or service-based access events on the basket picture. 

10 [0097] Meanwhile, the basket information may be 
managed by the viewer terminal or by the terminal of the 
server acting as an intermediary for the sponsorship 
transaction. When the basket information is managed 
by the terminal on the viewer side, the information con- 

15 cem ing the commodities or services the viewer has had 
the intention to purchase is transmitted to the server side 
at a stage when the will for purchase is re-confirmed. 
[0098] Meanwhile, the present program ancillary data 
processing device can be used irrespective of whether 

20 the device is of portable type or of a standstill type. That 
is, the device may be a computerterminal, such as note- 
book or desktop personal computer, a portable informa- 
tion terminal (PDA), a portable telephone set, a game 
machine, a set top box, a PC card, domestic electrical 

25 utensils, VTR, a television receiver, a car navigation sys- 
tem, a wrist watch having the information processing 
functions, or a remote controller, in addition to the de- 
vices dedicated to the processing of program ancillary 
data. 

30 [0099] It is noted that the explanation on other terms 
common to the above-described means is the same as 
that for these means. 

[0100] As the seventeenth means, the present inven- 
tion proposes a server device including an access infor- 

35 mation acquisition unit for acquiring the information per- 
tinent to the access state from the viewer to a database 
supervising the detailed information pertinent to com- 
modities or services appearing as an imaged object in 
a picture program, in accordance with a request from 

40 the viewer connected to the acquisition unit over a com- 
munication network, and a communication functional 
unit for notifying the latest value of the number of times 
of scene-based access, as acquired by the access in- 
formation acquisition unit, and as requested by the view - 

45 er. 

[0101] With use of this server device, it is possible to 
render services using the information concerning the ac- 
cess state from the viewer to the detailed information 
pertinent to commodities or services even when such 

50 information is not included in the picture program data 
received or reproduced by the viewer. Since the infor- 
mation supplied here is the number of times of scene- 
based access events, the viewer is able to judge wheth- 
er or not popular commodities or services are contained 

55 in the scene in question. This information also proves 
beneficial to the business undertaker who furnished the 
commodities or services to the program or brokerage 
business operator acting as intermediary for the spon- 



9 



17 



EP 1 335 592 A1 



18 



sorship transaction between the viewer and the busi- 
ness undertaker. 

[0102] Meanwhile, the 'database supervising the de- 
tailed information' not only may be a producer, a broad- 
caster or a distributor of the picture program but also s 
may be a producer ora distributor of a recording medium 
or a business undertaker acting as an intermediary for 
purchasing the commodities or services. Meanwhile 
the database superintendent may be the same as or dif- 
ferent from the server device superintendent. Of course 10 
the database may be provided in the server device itself' 
or may be connected to the server device over a net- 
work. 

[0103] The 'information pertinent to the access state' 
may be the information corresponding to the sum of the 15 
number of simple accesses to the detailed information 
pertinent to commodities or services which failed to lead 
to purchase and the number of times of access events 
which led to purchase. Alternatively, it may be the infor- 
mation pertinent to only one of the above two numbers so 
of times of access events. 

[0104] The Viewer 1 may be limited to the specified 
viewer who has concluded service contracts whether 
chargeable or non-chargeable, with the superintendent 
of the server device, or to the entire viewers that can be 25 
connected to the server device over a communication 
network. 

[0105] The 'server-device' simply has the meaning 
that the device operates as a server device with respect 
to the viewer terminal, and may be a computer terminal 30 
(notebook or desktop computer), a portable information 
terminal (PDA), a portable telephone set, a game ma- 
ch.ne, a set top box, a PC card, a car navigation system 
a wrist-watch having the information processing func- 
tion, and so forth, in addition to a dedicated server de- 35 
vice. 

[0106] It is noted that the explanation on other terms 
common to the above-described means is the same as 
that for these means. 

[0107] As the eighteenth means, the present inven- 40 
tion proposes a server including an access information 
acquisition unit for acquiring the information pertinent to 
the access state from the viewer to a database which 
supervises the detailed information pertinent to com- 
modities or services appearing as an imaged object in 4s 
a picture program, in accordance with a request from 
the viewer connected to the acquisition unit over a com- 
munication network, and a communication functional 
unit for notifying the latest value of the number of times 
of commodity-based or scene-based access, as ac- so 
quired by the access information acquisition unit, and 
as requested by the viewer. 

[0108] The difference between this server device and 
the server device of the seventeenth means is that the 
information furnished is the latest value of the number ss 
of times of commodity- or service-based access events 
in each scene. 

[0109] Thus, with this means, the latest value of the 



access states from the viewer to the detailed information 
pertinent to commodities or services appearing as an 
imaged object in the picture program can be incidentally 
supplied on request from the viewer. Meanwhile, since 
the information furnished here is the number of com- 
modity- or service-based access events for commodi- 
ties or services appearing as an imaged object in each 
scene, the viewer is able to judge easily which is the 
commodity or service now in vogue. This information al- 
so proves beneficial to the business undertaker who fur- 
nished the commodities or services to the program or 
brokerage business operator acting as intermediary for 
the sponsorship transaction between the viewer and the 
business undertaker. 

[0110] Meanwhile, the explanation on other terms 
common to the above-described means is the same as 
that for these means. 

[0111] It should be noted that, if the seventeenth 
means is combined with the eighteenth means, such a 
server device may be realized which is able to furnish 
the latest value of the number of times of scene-based 
access events and the latest value of the commodity- or 
service-based access events of the scene in question 
simultaneously. 

[0112] As nineteenth means, the present invention 
proposes a server device including an information fur- 
nishing unit for accessing a database supervising the 
detailed information relevant to commodities or services 
appearing as an imaged object in a picture program re- 
sponsive to a request from the viewer connected to'the 
information furnishing unit over a communication net- 
work, for furnishing a list file provided for presenting the 
information relevant to plural scenes in a list form to the 
viewer. The list file is an information file provided for the 
relevant picture program. 

[0113] With use of this server device, if an information 
file in question is not included in the picture program da- 
ta received or reproduced by the viewer, it is possible to 
enjoy services employing such file. 
[01 14] For each of the 'database', Viewer" and 'server 
device', what has been said of the database in the sev- 
enteenth or eighteenth means holds. 
[0115] Meanwhile, th e 'list file' may already exist in the 
database, or may be prepared by the server device 
based on the information supervised in the database. 
The range of the plural scenes making up the 'list file' 
may be decided on responsive to the setting by the su- 
perintendent of the database or the server device or an 
optimum value of the range may be provided responsive 
to the viewer's demand. Of course, the range can be 
automatically set based on a predetermined setting rule 
(rule similar to that for the aforementioned program an- 
cillary data processing device). Meanwhile, the ranges 
of the plural scenes making up the list file may or may 
not be overlapped with one another. 
[0116] Meanwhile, the 'list file' usually includes or is 
paired with the layout information. However, the layout 
information may not be furnished from the server device 
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in which case the display on the viewer side is based on 
the layout information provided on the viewer side. 
[0117] It is noted that the explanation on other terms 
common to the above-described means is the same as 
that for these means. 

[0118] Of course, the server device of the present 
nineteenth means may be combined with the function 
of furnishing the number of times of access events in 
the seventeenth or eighteenth means. When the seven- 
teenth means and the nineteenth means are to be com- 
bined together, the number of times of access events 
from one scene of the list file to another must be able to 
be furnished along with the list file. When the eighteenth 
means and the nineteenth means are to be combined 
together, the number of times of access events from one 
commodity or service for each scene of the list file to 
another must be able to be furnished along with the list 
file. Of course, the seventeenth means can be combined 
with the eighteenth means. 

[01 1 9] As the twentieth means, the present invention 
proposes the nineteenth means wherein the information 
furnishing unit has the function of furnishing an index 
picture associated with a scene in question for each 
scene forming the list file. 

[0120] With use of this server device, such a list file, 
each scene of which includes an index picture, can be 
provided. Since the index picture is attached to each of 
the scenes making up the list file, the operation of scene 
retrieval by the viewer is facilitated. 
[0121] Meanwhile, the 'index picture* may not only be 
provided on the side database but on the side server 
device. 

[0122] It is noted that the explanation on other terms 
common to the above-described means is the same as 
that for these means. 

[0123] As the twenty-first means, the present inven- 
tion provides a picture information displaying method for 
displaying, on a display picture, the information perti- 
nent to commodities or services appearing in each 
scene of a picture program, responsive to a request from 
the viewer, in which the method includes a processing 
step of reading out the results of summation of the 
number of times of scene-based access events of the 
viewer to a database, which supervises the detailed in- 
formation relevant to commodities or services appear- 
ing in a scene to be displayed, and a processing step of 
displaying the read-out results in association with the 
scene in question. 

[0124] With use of the program information displaying 
method, the viewer is able to judge readily whether or 
not popular commodities or services are included in a 
scene on the display picture, in which the viewer is in- 
terested, without regard to the route of acquisition of the 
information on the number of times of access events. In 
particular, the method is useful if, in browsing the infor- 
mation on the commodities or services appearing in im- 
aged objects in many scenes, a decision is to be given 
as to for which scene the information to be browsed. 



[0125] Meanwhile, the route of acquisition of 'the in- 
formation pertinent to commodities or services appear- 
ing in each scene' or the 'number of times of access 
events' may be the route of reception thereof, simulta- 

5 neously with the picture program data, route of repro- 
duction thereof, simultaneously with the picture program 
data, route of reception of the program ancillary data by 
itself, route of reproduction of the program ancillary data 
by itself, or the route of receiving the information from 

10 e.g., a database over a communication path. This dis- 
play function may be executed not only on a display pic- 
ture used for confirming the contents of the program an- 
cillary data but also on a basket picture or on a display 
picture for the detailed information of commodities or 

15 services. 

[0126] The terminal used for executing the method 
may be a computer terminal (notebook or desktop com- 
puter), PDA, a portable telephone set, a game machine, 
a set top box, a PC card, a car navigation system, a 

20 wrist- watch having the information processing function, 
and so forth, in addition to a dedicated server device. 
The respective steps of the above method do not have 
to be executed on the same device but may be executed 
on different devices. The same may be said for the 

25 methods that follow. 

[0127] Meanwhile, the explanation on other terms 
common to the above-described means is the same as 
that for these means. 

[0128] As the twenty-second means, the present in- 
30 vention proposes a picture information displaying meth- 
od for displaying, on a display picture, the information 
pertinent to commodities or services appearing in each 
scene of a picture program, responsive to a request from 
the viewer, in which the method includes a processing 
35 step of reading out the results of summation of the 
number of times of commodity-based or service-based 
access events by the viewer to a database, which su- 
pervises the detailed information relevant to the com- 
modities or services appearing in a scene to be dis- 
40 played, and a processing step of displaying the read-out 
results in association with each information relevant to 
the commodities or services appearing in a scene in 
question. 

[01 29] With use of this program information displaying 
45 method, the viewer is able to judge readily whether or 
not the specified commodities or services appearing in 
a picture program are in vogue among other viewers, 
irrespective of the routes of acquisition of the informa- 
tion on the number of times of access events. 
50 [0130] Meanwhile, the route of acquisition of 'the in- 
formation pertinent to commodities or services appear- 
ing in each scene' or the 'number of times of access 
events' may be the route of reception thereof simulta- 
neously with the picture program data, route of repro- 
55 duction thereof simultaneously with the picture program 
data, route of reception of the program ancillary data by 
itself, route of reproduction of the program ancillary data 
by itself, or the route of receiving the information from 
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e.g., a database over a communication path. This dis- 
play function may be executed not only on a display pic- 
ture used for confirming the contents of the program an- 
cillary data but also on a basket picture or on a display 
picture adapted for demonstrating the detailed informa- 
tion of commodities or services. 
[0131] Meanwhile, the explanation on other terms 
common to the above-described means is the same as 
that for these means. Of course, the step of displaying 
the number of times of access events on the commodity- 
or service-basis in the twenty-second means may be 
combined with the step of displaying the number of 
times of scene-based access events of the twenty-first 
means. 

[0132] As the twenty-third means, the present inven- 
tion proposes a picture information displaying method 
for displaying, on a display picture, the information per- 
tinent to commodities or services appearing in each 
scene of apicture program, responsive to a requestfrom 
the viewer, in which the method includes a processing 
step of reading out an index picture of a scene to be 
displayed, and a processing step of displaying the index 
picture, as read out, in conjunction with the information 
relevant to commodities or services appearing in the 
scene in question. 

[0133] With use of this program information displaying 
method, the viewer is able to judge readily which com- 
modrt.es or services appear in each scene, irrespective 
of the routes of acquisition of the information on the 
number of times of access events. In case of confirming 
the information pertinent to commodities or services the 
contents can be confirmed readily by displaying an in- 
dex picture, given in general as a still picture 
[0134] Meanwhile, the route of acquisition of the in- 
formation pertinent to commodities or services appear- 
ing in each scene' or the 'index picture' may be the route 
of acquisition of the information etc prepared in the de- 
vice for executing the method, such as the route of re- 
ception thereof simultaneously with the picture program 
data, route of reproduction thereof simuftaneously with 
the picture program data, route of reception of the pro- 
gram ancillary data by itself, route of reproduction of the 
program ancillary data by itself, or the route of having 
the information furnished from e.g., a database over a 
communication path. This display function may be exe- 
cuted not only on a display picture used for confirming 
the contents of the program ancillary data but also on a 
basket picture or on a display picture for the detailed 
information of commodities or services. 
[0135] Meanwhile, the explanation on other terms 
common to the above-described means is the same as 
that for these means. Of course, if the above means is 
combined with the twenty-first or twenty-second means 
such a display picture is obtained which is more con- 
venient to use for the user. 

[0136] As the twenty-fourth means, the present inven- 
tion proposes a program information displaying method 
for displaying on a display picture the information rele- 



vant to commodities or services appearing in each 
scene of a picture program, responsive to the viewer's 
request, in which the method includes a processing step 
of reading out the information relevant to the commod- 
5 ities or services appearing in each scene, for displaying 
the information relevant to plural scenes in a list form 
and a processing step of deploying the information, as 
read out, based on the preset layout information, for 
presentation in a list form on a display picture. 
" [0137] With use of the program information displaying 
method, in which the information relevant to plural 
scenes can be confirmed at a time on the display picture 
rt is unnecessary to perform the operation of displaying 
the information of another scene on the display picture 
even if the target information is not contained in the in- 
formation pertinent to a given scene. This enables effi- 
cient retrieval as compared to the case of individually 
accessing each picture. It should be noted that the list 
form picture may be a picture downloaded from another 
terminal device, or prepared in a terminal which exe- 
cutes the display method. 

[0138] For each scene displayed in a list form the 
number of times of scene-based access events or that 
of commodity- or service-based access events may be 
25 displayed. In this case, the above information may be 
directly downloaded from an external database. 
[0139] Meanwhile, the explanation on other terms 
common to the above-described means is the same as 
that for these means. 

30 [0140] As the twenty-fifth means, the present inven- 
tion proposes the twenty-fourth means where an index 
picture is displayed in each of the plural scenes in as- 
sociation therewith. 

[0141] With use of this program information displaying 
35 method, the target scene may be selected as the con- 
tents of each scene are confirmed with an index picture 
thereby improving the operating efficiency. When the in- 
formation pertinent to commodities or services is to be 
confirmed, its contents can be visually recognized on an 
4 0 index picture. 

[0142] Meanwhile, the explanation on other terms 
common to the above-described means is the same as 
that for these means. 

[0143] As the twenty-sixth means, the present inven- 
tion proposes a computer-readable recording medium 
having a program recorded thereon, the program being 
usable for a computer to implement a program ancillary 
data extraction function of extracting from picture pro- 
gram data program ancillary data relevant to commod- 
ities or services appearing in each scene, and a display 
control function of displaying, on a display picture the 
information relevant to the commodities or services ap- 
pearing in each scene, and the sum total of the number 
of times of scene-based access events by the viewer to 
55 a database supervising the detailed information relevant 
to the commodities or services, based on the preset lav- 
out information. 

[0144] With use of this recording medium, any suita- 
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ble pre-existing terminal can be used as a program an- 
cillary data processing device, on simply installing the 
program recorded on the recording medium on a pre- 
existing terminal, even if there is no device dedicated to 
program ancillary data. s 
[0145] The 'pre-existing terminal' may be a computer 
terminal (notebook or desktop computer), a portable in- 
formation terminal (PDA), a portable telephone set, a 
game machine, a car navigation system, a wrist-watch 
having the information processing function, and so forth. 
The recording medium may be a CD-ROM, a medium 
of the magnetic recording system, such as magnetic 
tape, floppy disc, hard disc or magnetic card, an optical 
recording type medium, such as write-once CD, DVD or 
an optical card, a magneto-optical recording type medi- 
um, such as DVD, MD or MO, or a semiconductor mem- 
ory, such as non-volatile memory or RAM. The same 
may be said of terminals or recording mediums in re- 
spective means that follow. 

[0146] Meanwhile, the explanation on other terms 
common to the above-described means is the same as 
that for these means. Although the program is assumed 
to be distributed by a recording medium, it may also be 
distributed over a network (transmission medium). 
[0147] As the twenty-seventh means, the present in- 
vention proposes a computer-readable recording medi- 
um having a program recorded thereon, the program be- 
ing usable for a computer to implement a program an- 
cillary data extraction function of extracting, from picture 
program data, program ancillary data relevant to com- 
modities or services appearing in each scene, and a dis- 
play control function of displaying, on a display picture, 
the information relevant to the commodities or services 
appearing in each scene, and the sum total of the 
number of times of commodity-based or service-based 
access events by the viewer to a database supervising 
the detailed information relevant to the commodities or 
services, based on the preset layout information. 
[0148] With use of this recording medium, any suita- 
ble pre-existing terminal can be used as a program an- 
cillary data processing device, on simply installing the 
program recorded on the recording medium on the ter- 
minal, even if there is no device dedicated to program 
ancillary data. 

[0149] Meanwhile, the explanation on other terms 
common to the above-described means, such as 'pre- 
existing terminal' or 'recording medium' is the same as 
that for these means. 

[0150] As the twenty-eighth means, the present in- 
vention proposes a computer-readable recording medi- 
um having a program recorded thereon, the program be- 
ing usable for a computer to implement a program an- 
cillary data extraction function of extracting from picture 
program data program ancillary data relevant to com- 
modities or services appearing in each scene, and a dis- 
play control function of displaying, on a display picture, 
the information relevant to the commodities or services 
appearing in each scene, based on the layout informa- 



tion adapted for presenting the information pertinent to 
plural scenes in a list form to the viewer. 
[0151] With use of this recording medium, any suita- 
ble pre-existing terminal can be used as a program an- 
cillary data processing device, on simply installing the 
program recorded on the recording medium on the ter- 
minal, even if there is no device dedicated to program 
ancillary data. 

[0152] Meanwhile, the explanation on other terms 
common to the above-described means, such as 'pre- 
existing terminal' or 'recording medium' is the same as 
that for these means. 

[01 53] As the twenty-ninth means, the present inven- 
tion provides a computer-readable recording medium 
having a program recorded thereon, in which the pro- 
gram is usable for a computer to implement an extrane- 
ous data presence/absence detection function of ex- 
tracting, from picture program data, data indicating 
whether or not the information relevant to commodities 
or services appearing in each scene is available through 
an external database, to detect the presence/absence 
of the extraneous data based on the so extracted data, 
a downloading function of downloading, when an ac- 
cess request to extraneous data is made by an operat- 
ing input by the viewer, a list file which is an information 
file provided from the outset for a relevant picture pro- 
gram for presenting the information relevant to plural 
scenes to the viewer in a list form, from an external da- 
tabase, and a display control function of opening the 
downloaded list file to display the information relevant 
to the plural scenes involved in the list file on a display 
picture in a list form, based on the preset layout infor- 
mation. 

[0154] With use of this recording medium, any suita- 
ble pre-existing terminal can be used as a program an- 
cillary data processing device, on simply installing the 
program recorded on the recording medium on the ter- 
minal, even if there is no device dedicated to program 
ancillary data. 

[0155] Meanwhile, the explanation on other terms 
common to the above-described means, such as 'pre- 
existing terminal' or 'recording medium* is the same as 
that for these means. 

[0156] As the thirtieth means, the present invention 
proposes a computer-readable recording medium hav- 
ing a program recorded thereon, the program being us- 
able for a computer to implement a downloading func- 
tion of downloading the information relevant to commod- 
ities or services appearing in each scene from a record- 
ing medium supervised by another terminal device, by 
communication with another terminal device, and a dis- 
play control function of demonstrating the information 
relevant to commodities or services appearing in each 
scene, on a display picture, responsive to an operating 
input by an operator. The information relevant to com- 
modities or services is program ancillary data of a pic- 
ture program in question. 

[0157] With use of this recording medium, any suita- 
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b e pre-existing terminal can be used as a program an- 
cillary data processing device, on simply installing the 
program recorded on the recording medium installed on 
the terminal, even if there is no device dedicated to pro- 
gram ancillary data. 5 
[0158] Meanwhile, the explanation on other terms 
common to the above-described means, such as 'pre- 
existing terminal' or 'recording medium' is the same as 
that for these means. 

[0159J As the thirty-first means, the present invention w 
proposes a computer-readable recording medium hav- 
ing recorded thereon a program usable for a computer 
to implement a display control function of displaying an 
index picture of a scene, in an imaged object of which 
are included commodities or services selected with a will 13 
forpurchasing, in a correlated fashion in a basketpicture 
m which the will for purchasing can be confirmed prior 
to final transmission to the server side. 
[0160] With use of this recording medium, any suita- 
ble pre-existing terminal can be used as a program an- so 
ciliary data processing device, on simply installing the 
program, recorded on the recording medium, on the ter- 
minal, even if there is no device dedicated to proqram 
ancillary data. y 
[0161] Meanwhile, the explanation on other terms 25 
common to the above-described means, such as 'pre- 
ex,sting terminal' or 'recording medium' is the same as 
that for these means. 

[01 62] As the thirty-second means, the present inven- 
tion proposes a computer-readable recording medium so 
having recorded thereon a program usable for a com- 
puter to implement an access information acquiring 
function of acquiring, responsive to a request from the 
viewer connected to the computer over a communica- 
tion network, the information pertinent to access states 35 
from the viewer to a database supervising the detailed 
information pertinent to commodities or services ap- 
pearing as an imaged object in a picture program and 
a communication function for notifying the latest value 
of the number of times of scene-based access events 40 
as acquired by theaccess information acquiring function 
and as requested by the viewer. 
[0163] With use of this recording medium, any suita- 
ble pre-existing terminal can be used as a program an- 
cillary data processing device, on simply having the pro- 45 
gram recorded on the recording medium installed on the 
terminal, even if there is no device dedicated to proqram 
ancillary data. 

[0164] Meanwhile, the explanation on other terms 
common to the above-described means, such as 'pre- so 
existing terminal' or 'recording medium' is the same as 
that for these means. 

[0165] As the thirty-third means, the present invention 
proposes a computer-readable recording medium hav- 
ing recorded thereon a program usable for a computer ss 
to implement an access information acquiring function 
of acquiring, responsive to a request from the viewer 
connected to the computer over a communication net- 



work, the information pertinent to access states from the 
viewer to a database supervising the detailed informa- 
tion pertinent to commodities or services appearing as 
an imaged object in a picture program, and a communi- 
cation function for notifying the latest value of the 
number of times of community-based or service-based 
access events as acquired by the access information 
acquiring function and as requested by the viewer. 
[0166] With use of this recording medium, any suita- 
ble pre-existing terminal can be used as a program an- 
cillary data processing device, on simply having the pro- 
gram recorded on the recording medium installed on the 
terminal, even if there is no device dedicated to program 
ancillary data. y 
[0167] Meanwhile, the explanation on other terms 
common to the above-described means, such as 'pre- 
existing terminal' or 'recording medium' is the same as 
that for these means. 

[0168] As the thirty-fourth means, the present inven- 
tion proposes a computer-readable recording medium 
having recorded thereon a program usable for a com- 
puter to implement an information furnishing function of 
accessing a database supervising the detailed informa- 
tion pertinent to commodities or services appearing as 
an imaged object in a picture program, responsive to a 
request from the viewer connected to the computer over 
a communication network, for presenting a list file pro- 
vided from the outset for presenting the information rel- 
evant to plural scenes to the viewer in a list form The 
list file is an information file provided from the outset for 
a relevant picture program. 

[0169] With use of this recording medium, any suita- 
ble pre-existing terminal can be used as a program an- 
cillary data processing device, on simply having the pro- 
gram recorded on the recording medium installed on the 
terminal, even if there is no device dedicated to proqram 
ancillary data. y 
[0170] Meanwhile, the explanation on other terms 
common to the above-described means, such as 'pre- 
existing terminal' or 'recording medium' is the same as 
that for these means. 

Brief Description of the Drawings 
[0171] 

Fig. 1 shows a conceptual structure of an entire dig- 
ital broadcast system. 

Fig.2 shows a conceptual structure of a digital 
broadcast system in the perspective of circulation 
of data ancillary to a program. 
Fig.3 shows an illustrative structure of a digital 
broadcasting system. 

Fig.4 is a functional block diagram showing an em- 
bodiment of a sponsorship request condition input- 
ting device (first embodiment). 
Fig.5 is a functional block diagram showing an em- 
bodiment of a sponsorship request condition input- 
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ting device (second embodiment). 
Fig. 6 shows an embodiment of a display picture for 
a sponsorship request condition inputting form. 
Fig.7shows an illustrative structure of sponsorship 
request condition data. 5 
Fig.8 is a flowchart showing an embodiment of a 
processing sequence in inputting the sponsorship 
request condition. 

Fig.9 is a functional block diagram showing an em- 
bodiment of a response result receiving device. 10 
Fig.1 0 shows an embodiment of a display picture of 
a form of response result confirmation. 
Fig. 11 shows an illustrative structure of response 
result data. 

Fig.1 2 is afunctional block diagram showing an em- is 
bodiment of a sponsorship transaction brokerage 
server. 

Fig.1 3 shows an illustrative structure of data of a 
membership database. 

Fig.1 4 is a flowchart showing a typical processing 20 
sequence on receipt of a sponsorship request. 
Fig. 1 5 is a functional block diagram showing an em- 
bodiment of a sponsorship request receiving de- 
vice. 

Fig.1 6 shows an embodiment of a display picture of 25 
a form of confirmation of request content. 
Fig. 1 7 is a functional block diagram showing an em- 
bodiment of an advance registration information in- 
putting device. 

Fig.1 8 shows an embodiment of a display picture of 30 
a generic item registration form. 
Fig.1 9 shows an embodiment of a display picture of 
a sponsorship condition registration form. 
Fig.20 shows an illustrative structure of ancillary in- 
formation data. 35 
Fig.2 1 shows a specified embodiment of commodity 
and service data in agreement with the content of a 
video program. 

Fig.22 is afunctional block diagram showing an em- 
bodiment of a device for the preparation of data an- 40 
ciliary to a program (first embodiment). 
Fig.23 is afunctional block diagram showing an em- 
bodiment of a device for the preparation of data an- 
cillary to a program (second embodiment). 
Fig.24 is a functional block diagram showing an em- 45 
bodiment of a device for the preparation of data an- 
cillary to a program (third embodiment) 
Fig.25 shows an embodiment of a display picture of 
a form for inputting data ancillary to a program (first 
embodiment). so 
Fig.26 shows an embodiment of a display picture of 
a form for inputting data ancillary to a program (sec- 
ond embodiment). 

Fig.27 shows an embodiment of a display picture of 

a form for inputting data ancillary to a program (third ss 

embodiment). 

Fig.28 is a flowchart showing an embodiment of a 
processing sequence in inputting data ancillary to a 



program (first embodiment). 
Fig. 29 is a flowchart showing an embodiment of a 
processing sequence in inputting data ancillary to a 
program (second embodiment). 
Fig. 30 is afunctional block diagram showing an em- 
bodiment of a device for the preparation of data an- 
cillary to a program. 

Fig.31 is afunctional block diagram showing an em- 
bodiment of a device for the preparation of picture 
program data. 

Fig. 32 is afunctional block diagram showing an em- 
bodiment of a video program editing device. 
Fig. 33 shows an embodiment of a display image in 
preparing or editing data ancillary to a program (first 
embodiment). 

Fig. 34 shows an embodiment of a display image in 
preparing or editing data ancillary to a program 
(second embodiment). 

Fig. 35 shows an embodiment of a display image in 
preparing or editing data ancillary to a program 
(third embodiment). 

Fig. 36 shows an embodiment of a display image in 
preparing or editing data ancillary to a program 
(fourth embodiment). 

Fig.37 shows an embodiment of a display image in 
preparing or editing data ancillary to a program (fifth 
embodiment). 

Fig.38 is a flowchart showing an embodiment of a 
processing sequence in preparing data ancillary to 
a program (first embodiment). 
Fig. 39 is a flowchart showing an embodiment of a 
processing sequence in preparing data ancillary to 
a program (second embodiment). 
Fig.40 is afunctional block diagram showing an em- 
bodiment of a device for processing data ancillary 
to a program (first embodiment). 
Fig. 41 is a functional block diagram showing an em- 
bodiment of a device for processing data ancillary 
to a program (second embodiment). 
Fig. 42 is afunctional block diagram showing an em- 
bodiment of a device for processing data ancillary 
to a program (third embodiment). 
Fig. 43 is afunctional block diagram showing an em- 
bodiment of a device for processing data ancillary 
to a program (fourth embodiment). 
Fig.44 is afunctional block diagram showing an em- 
bodiment of a device for processing data ancillary 
to a program (fifth embodiment). 
Fig. 45 shows an embodiment of a display picture of 
a video program provided with data ancillary to a 
program (first embodiment). 
Fig. 46 shows another embodiment of a display pic- 
ture of a video program provided with data ancillary 
to a program (second embodiment). 
Fig.47 shows an embodiment of a display picture of 
a meta-pocket picture (first embodiment). 
Fig. 48 shows an embodiment of a display picture of 
a meta-pocket picture (second embodiment). 
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Fig.49 shows an embodiment of a display picture of 
a meta-pocket picture (third embodiment). 
Fig.50 shows an embodiment of a display picture of 
a meta-pocket picture (fourth embodiment). 
Fig.51 shows an embodiment of a display picture of s 
a meta-mall picture (first embodiment). 
Fig.52 shows an embodiment of a display picture of 
a meta-mall picture (second embodiment). 
Fig.53 shows an embodiment of a display picture of 
a meta-cart picture. ' 10 

Fig.54 shows an embodiment of a commodity infor- 
mation site picture. 

Fig.55 shows an embodiment of another display 
picture (first embodiment). 

Fig.56 shows an embodiment of still another display is 
picture (second embodiment). 
Fig.57 is a flowchart showing an example of a 
processing sequence in displaying data ancillary to 
a program. 

Fig.58 is a functional block diagram showing an em- 20 
bodiment of a server device (first embodiment) 
Fig.59 is a functional block diagram showing an em- 
bodiment of a server device (second embodiment) 
Fig.60 is a flowchart showing an example of a 
processing sequence in displaying a meta-pocket 25 
Fig.61 is a flowchart showing an example of a 
processing sequence in displaying a meta-mall 



Best Mode for Carrying out the Invention 

[01 72] Referring to the drawings, certain present em- 
bodiments of the present invention will be explained in 
detail. 

(A) Overall System Envisioned in the Present 
Specification 

(A-1) Example of Conceptual Structure 

[0173] First, an example of a conceptual structure of 
the entire system, as envisioned in the present specifi- 
cation, is explained. The entire system includes as a 
basic structure, a stage of sponsorship tie up to a project 
(stage of receiving the accepting a project), a stage of 
employing goods or services furnished to execute the 
project, including creation and distribution, and a stage 
of coupling the results of execution to on-line shopping 
[0174] In the ensuing explanation, it is assumed that 
the 'project 1 is the creation of a digital broadcast pro- 
gram. Fig.1 shows an illustrative conceptual structure of 
the digital broadcast system. Of course, Fig.1 shows an 
illustration, such that variegated forms may be con- 
ceived for the digital broadcasting system. 
[01 75] The digital broadcasting system, shown in Fig 
1 , is roughly made up of three partial systems. One of 
these systems is a sponsorship transaction support sys- 
tem for on-line realization of a transaction pertinent to a 
sponsorship request of goods and services to a digital 



broadcast program. Another of the systems is a subsid- 
iary information data creating system for preparing pro- 
gram ancillary data, which correlates the information 
pertinent to the commodities and services offered by the 
sponsor business operators to each scene of the picture 
program, while the remaining one is the program ancil- 
lary data processing system for receiving or reproducing 
the program ancillary data correlated to each scene to 
realize on-line shopping. 

[0176] In the case of Fig.1 , the sponsorship transac- 
tion support system is made up by a sponsor business 
operator 1, a broadcast business operator 2 and by a 
sponsorship transaction brokerage business operator 4 
The subsidiary information data creating system is 
made up by the broadcast business operator 2 and by 
a shopping brokerage business operator 5. The pro- 
gram ancillary data processing system is made up by 
the audience 3, the shopping brokerage business oper- 
ator 5 and by the sponsor business operator 1 
[0177] It should be noted that the presence of broker- 
age business operators in each partial system is not 
mandatory, that is, if there is no interposing brokerage 
business operator, the functions necessary for a trans- 
action are executed by one or more of the sponsor busi- 
ness operator 1 , broadcast business operator 2 and the 
audience 3. 

[0178] Although the sponsorship transaction broker- 
age business operator 4 and the shopping brokerage 
business operator 5 are routinely managed by different 
undertakers, there may, of course, be managed by the 
same undertaker. 

(a) sponsorship transaction support system 

55 [0179] In the case of the sponsorship transaction sup- 
port system, shown in Fig. 1 , an entity making a request 
for sponsorship for creation of a video program is the 
broadcast business operator 2, while the undertaker 
ready to accept the request is the sponsor business op- 
erator 1. It is the sponsorship transaction brokerage 
business operator 4 that acts as intermediary for the 
sponsorship transaction. 

[0180] The sponsorship transaction routinely is exe- 
cuted as follows: A broadcast business operator first is- 
sues a request for sponsorship to the brokerage busi- 
ness operator. Of course, this request for sponsorship 
is made on-line. The brokerage business operator for- 
wards a request for sponsorship, accepted from the 
broadcast business operator, to sponsor business op- 
erators registered in a database (registered members) 
[0181] Variegated methods may be thought of as the 
forwarding method. Examples of these methods include 
sending the request unconditionally to all registered 
members, and sending the request selectively based on 
55 the content of the request for sponsorship. The latter 
method may be classified into a method of taking into 
account the conditions as set by the broadcast business 
operator, as a client, a method of taking into account the 
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conditions as set by the broker acting as intermediary 
for sponsorship request, a method of taking into account 
the conditions as set by the sponsor business operator, 
as a request receiving party, and a method comprised 
. of optional combinations of these respective methods. 
[01 82] This forwarding is accompanied by a metadata 
frame (right for making insertions in the data ancillary to 
the program). That is, the forwarding is accompanied by 
the right to get the information pertinent to furnished 
goods or services included in metadata (data ancillary 
to the program) which is the ancillary information to pro- 
gram data (generally video and acoustic data and occa- 
sionally the video data or acoustic data) if the member 
who has received the request agrees to the sponsor- 
ship. 

[0183] The sponsor business operator, who has re- 
ceived and is willing to agree to the request, notifies the 
brokerage business operator of such effect. The broker- 
age business operator checks whether or not the con- 
tent of the agreement conforms to items of request. 
When the content of the agreement conforms to the 
items of request, for example, when the request desti- 
nation is only one business operator and only one busi- 
ness operator has agreed to the sponsor, or when the 
quantity of the goods to be furnished is in meeting with 
the quantity as stated in the request, the brokerage busi- 
ness operator notifies the broadcast business operator 
of the content of the finalized transaction. 
[0184] When the content of the acceptance is not 
quantitatively up to the items of sponsorship, for exam- 
ple, when the number of the business operators who 
agreed to the sponsorship exceed a scheduled number 
or when the number of goods that can be furnished by 
the agreeing plural business operators exceeds a 
scheduled number, the brokerage business operator 
specifies the quantity for the benefit of the broadcast 
business operator, or adjusts the number of goods fur- 
nished by the agreeing business operators, and notifies 
the broadcast business operator of the content of the 
finally determined transaction. If necessary, the ex- 
change of communication may take place several times 
between the brokerage business operator and the spon- 
sor business operators). 

[0185] Meanwhile, if the agreed content is not quan- 
titatively up to the items of sponsorship, for example, if 
the conditions in the agreed content are not up to the 
desired conditions geographically or temporally, the bro- 
kerage business operator notifies the sponsor business 
operator of the effect of failure in the conclusion of the 
transaction. Meanwhile, if the sponsorship transaction 
is not concluded, the brokerage business operator no- 
tifies the broadcast business operator of the failure in 
the conclusion of the sponsorship transaction, from one 
subject-matter of the transaction to another, that is from 
one commodity or service to another. 
[0186] The foregoing is the general flow of the spon- 
sorship transaction. Meanwhile, if an operator with high 
possibility of success in the conclusion of the sponsor- 



ship transaction has become apparent prior to transmis- 
sion of the sponsorship request by the brokerage busi- 
ness operator, it may be contemplated to inform only the 
parties concerned of the existence of the competing 

5 counterparty. This may be realized by pre-registration 
on the database by no other than the operator willing to 
agree to the sponsorship of the conditions under which 
the operator may agree to the sponsorship. In such 
case, the brokerage business operator proceeds to 

10 match the content of the sponsorship request received 
from the client and the information of advance registra- 
tion in the database to finally decide on the parties of 
contract. 



[0187] In the case of the subsidiary information data 
creating system, shown in Fig. 1 , it is the broadcast busi- 
ness operator 2 who creates and distributes a video pro- 

20 gram, while it is the shopping brokerage business oper- 
ator 5 who creates program ancillary data to be associ- 
ated with respective scenes of the picture program. 
Meanwhile, it may be an occurrence that the sponsor- 
ship transaction brokerage business operator 4 acing 

25 as intermediary for sponsorship transactions becomes 
an entity in creating the program ancillary data, or the 
broadcast business operator 1 for itself becomes an en- 
tity in creating the program ancillary data. 
[0188] Meanwhile, the broadcast business operator 

30 may encompass a company specialized in creating an 
image program. Today, the broadcast business operator 
may even be consumers in consideration of the devel- 
opment in the communication technology and cost re- 
duction in variegated equipment. 

35 [0189] The creation and distribution of picture pro- 
grams at large may be achieved as follows: First, in cre- 
ating, that is the imaging, storage or the editing etc. of 
the picture program, the broadcast business operator, 
usually made up by a creating company and a distribut- 

40 jng company, prepares and records program ancillary 
data (metadata) relevant to each scene of a picture pro- 
gram. The program ancillary data (metadata) here is 
termed key data at large or management data. These 
are data necessary for e.g., data management. The pro- 

45 gram ancillary data is also used for retrieving data from 
variegated stored information. For example, the meta- 
data may be exemplified by the code information, such 
as UMID (Unique Material Identifier), capable of unique- 
ly identifying the image material on a global extent, such 

so as on the station-wide, keiretuwide, or nation-wide ba- 
sis, scene numbers, performers, positions, sites, such 
as GPS information and by program names. It is how- 
ever desirable to be able to use the code information 
that can be identified uniquely on the global basis. 

55 [0190] Moreover, data of commodities and services 
for the respective scenes are also recorded. By and 
large, these information are not necessarily limited to 
sponsor goods. However, since the data on the com- 
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modrt.es and services are also utilized in the present 
specification for on-line shopping, it is necessary that 
only the data relevant to the sponsorship (subject-mat- 
ter of sponsorship) are recorded or that distinction may 
be made between the goods or services as the subject- 
matter of the sponsorship and those which are not the 
subject-matter of the sponsorship. 
[0191] In the case of Fig.1, only the minimum infor- 
mation recorded in the course of imaging, such as the 
names of the sponsor business operators or the generic 
names of commodities and services, for example spec- 
tacles, retail, etc., are given the broadcast business op- 
erator who finally creates the program ancillary data 
(metadata). Of course, the broadcast business operator 
may create data up to precise details, or no data con- 
cerning the commodities or services may be recorded 
depending on the type of the system under construction' 
[01 92] The brokerage business operator collects fur- 
ther detailed information concerning the commodities or 
services, notified by the broadcast business operator 
from the sponsor business operators, or from the data- 
base, which may be supervised by the sponsorship 
transaction brokerage business operator or by each 
sponsor business operator. The broadcast business op- 
erator then creates a picture to be furnished to the au- 
dience, using the description language, that is BML 
(broadcast markup language) or XML (extensible 
markup language), including the layout information (lay- 
out information in a picture), to send the so created im- 
age information as program ancillary data to the broad- 
cast business operator. 

[0193] Of course, there may be an occasion wherein 
such program ancillary data is sent to the broadcast 
business operator in which the layout information (tem- 
plate image indicating the pasting position for the tag 
and the information providing for the method for repre- 
sentation) and the information concerning the commod- 
ities or services as collected represent different data 
[01 94] That is, the program ancillary data (metadata) 
may be classified into one comprised of the layout infor- 
mation and the data of the commodities or services uni- 
fied together, that is data finished to such an extent that 
the data can directly be viewed or heard on expansion 
m accordance with the description language, and one 
in which the layout information is distinct from the data 
on commodities or services, that is data in which, in data 
presentation to the user, the data on commodities or 
services are pasted to the layout information. 
[0195] Moreover, the brokerage business operator 
creates the index information, usable for representing 
the respective scenes associated with the program an- 
cillary data in a simplified fashion, as necessary and 
sends the so created index information to the broadcast 
business operator. 

[0196] In any of these cases, the brokerage business 
operator adds the code information, specifying the re- 
spective scenes, to the program ancillary data. Based 
on this code information, the broadcast business oper- 



ator is able to reliably correlate the images of the asso- 
ciated scenes with the program ancillary data. 
[0197] On acquisition of the program ancillary data 
from the brokerage business operator, the broadcast 
business operator correlates the program data and the 
program ancillary data with each other, based on the 
code information, and transmits the resulting data as 
broadcast waves in an ultimately output form. Although 
Rg.1 shows the case of satellite broadcast, the broad- 
10 cast may be by ground waves or waves transmitted over 
a cable, such as power line, telephone line, optical cable 
etc. The output form may, of course, be communication 
in place of broadcast. The data may also be recorded 
on a package medium for distribution as a package me- 
is dium. 

(c) Digital picture program receipt/reproducing system 



[0198] In the case of the digital picture program re- 
ceiving/reproducing system, shown in Fig.1 , it is the au- 
dience 3 that receives or reproduces the picture pro- 
gram, while it is the shopping brokerage business oper- 
ator 5 that accepts the offer of the detailed commodity 
information responsive to the request from the audi- 
ts ence, or accepts the order. On the other hand it is the 
sponsor business operator 5 that furnishes the com- 
modities or services based on the buy-sell agreement 
concluded with the audience. 

[0199] There may, of course, be a system devoid of 
30 an interposing shopping brokerage business operator. 
There may also be an occasion where no information 
concerning the commodities or services is receivedfrom 
the brokerage business operator in case the information 
on the commodities or services given as program ancil- 
35 lary data (metadata) suffices. 

[0200] The viewing of the picture programs at large 
and the on-line shopping occur as follows: If, in viewing 
the picture program, the audience comes across goods 
or services he or she is interested in, he or she operates 
40 for demonstrating the program ancillary data, usually by 
acting on a remote controller. When there is any pro- 
gram ancillary data associated with the scene, the infor- 
mation is displayed on part or all of the display picture 
[0201] Usually, a display image representing the in- 
formation concerning the commodities or services is 
represented. In the present specification, this informa- 
tion is referred to as meta-pockets or meta-mall The 
meta-pocket is used in the meaning of a display picture 
for the information concerning the commodities or serv- 
ices appearing in a particular scene or the program an- 
cillary data associated with the information, while the 
meta-mall is used in the meaning of a display picture 
which allows for overview of meta-pockets for plural 
scenes on a sole picture or in the meaning of the pro- 
55 gram ancillary data associated with the information. 
[0202] These display pictures may be demonstrated 
simultaneously with the display of a picture program as 
the main program, or independently of the picture pro- 
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gram display, such as on termination of the broadcast 
or reproduction of the picture program. 
[0203] As for the content of the display picture, solely 
the information concerning the sole commodities or 
services represented on the display picture may be dis- 
played, or the totality of the information concerning the 
plural commodities or services represented on the sole 
picture may be displayed. Although it is assumed in Fig. 
1 that the commodities or services to be demonstrated 
are supplied by the sponsor business operator, the com- 
modities or services supplied may also be non-sponsor- 
ship commodities or services (commodities or services 
which are not the subject-matter of the sponsorship). 
The commodities or services demonstrated may not be 
sponsorship commodities or services (commodities or 
services which are the subject-matter of the sponsor- 
ship) but may be the commodities or services of the pro- 
gram purveyors. 

[0204] The display content may also be variable, as 
may be seen from the specified examples as later ex- 
plained. It is assumed that the audience has requested 
the display of the information which is more detailed 
than the information currently displayed on the display 
picture. When the request is input by a specified actua- 
tion by the audience, the information distribution is re- 
quested to a brokerage business operator from a termi- 
nal fitted with a communication function and which forms 
a viewing system along with a display device, routinely 
a television receiver or a set top box. The uplink trans- 
mission path, over which the request is transmitted, may 
be exemplified by the telephone network, packet net- 
work or the Internet. The transmission medium may, of 
course, be cabled or wireless. 

[0205] On receipt of the request from the audience, 
the brokerage business operator reads out the informa- 
tion as requested from the database to transmit the so 
read-out information. This information may include the 
information indicating the degree of interest on the part 
of the audience, such as number of times of access 
events, in the scene in question, and the commodities 
or services appearing in the scene. It is among the ad- 
vantages of providing the brokerage business operator 
to be able to supply this sort of the information. 
[0206] If subsequently the audience is desirous to 
purchase the commodity he or she is interested in, he 
or she sends order data, as the expression of such in- 
tention, over the uplink transmission path to complete 
the buy-seli agreement with the sponsor business oper- 
ator. Before finally transmitting order data, the state of 
the order may be confirmed by a basket picture, as in 
the case of the currently operating Internet shopping. 
This basket picture, also termed a cart picture, enables 
the subject-matter purchased to be re-confirmed before 
actually sending the intention of buying the goods to the 
sponsor business operator. 

[0207] When the intention of the audience in buying 
a commodity is notified to the sponsor business opera- 
tor, shipment of the commodity and payment are per- 



formed subsequently as in the routine commercial trans- 
actions. If the information on the picture program or the 
scene which acted as incentive for acquiring the infor- 
mation on the commodity or in placing an order is in- 

5 eluded in the communication the audience has with the 
brokerage business operator or with the sponsor busi- 
ness operator, in a manner not shown in Fig.1 such in- 
cluded information may be used as a reference material 
at the time of scheming a new picture program or at the 

10 time of accepting the sponsorship. 

(d) Flow of program ancillary data 

[0208] Fig. 2 shows an illustrative conceptual struc- 
15 ture of the digital broadcast system as viewed from the 
flow of the program ancillary data. In Fig. 2, the digital 
broadcast system is roughly classified into four partial 
systems, namely a sponsorship transaction brokerage 
system 6, a creation broadcast system 7, a reception 
20 reproduction system 8 and a shopping brokerage sys- 
tem 9, with the types of handling of the program ancillary 
data as ad-hoc reference or index. In Fig. 2, the sponsor 
business operator belongs mainly to the sponsorship 
transaction brokerage system and the audience be- 
25 longs to the reception reproduction system. 

[0209] The feedback information (access information, 
agreement conclusion information or management da- 
ta), as collected from the audience who viewed a picture 
program, is sent from the shopping brokerage system 9 
30 to the sponsorship transaction brokerage system 6. As 
the access information, there may, for example, be the 
information as to from which scene of which picture pro- 
gram and for which commodities or services the access 
was made, or the information as to the sex or the age 
35 group of the audience. The information as to the con- 
clusion of agreements may be the number of concluded 
buy-sell agreements or the content of the agreements. 
The management data may be the program names, 
broadcast date and time and program frames. 
40 [0210] In the sponsorship transaction brokerage sys- 
tem 6, the above information is used as the reference 
materials for scheming new picture programs or for fu- 
ture sponsorship in order to execute a sponsorship 
transaction for a new picture program. The information 
45 finally determined in this transaction, for example, the 
names of the establishments as participants in the spon- 
sorship or the information on the commodities or serv- 
ices as the subject-matter of the sponsorship, is sent to 
the creation broadcast system 7. 
so [0211] The creation and broadcasting system 7 su- 
pervises the variegated data produced in the course of 
creation of picture programs, such as scheming data, 
imaging data, storage data, editing data and distribution 
data, correlated with the information supplied from the 
55 sponsorship transaction brokerage system 6 or from the 
shopping brokerage system. 

[021 2] The scheming data may be enumerated by, for 
example, the tenor of program creation, target, schem- 
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mg staff members etc. The imaging data may be enu- 
merated by, for example, the date and time, site or 
weather at the time of imaging, staff members, sponsor 
goods or services used in each scene, and imaging 
equipment. The storage data may, for example, be the 
identification number and address of a recording medi- 
um. The editing data may, for example, be the editing 
scene, special effects used, editing staff members etc 
The distribution data may, for example, be the date and 
time of distribution, territory, copyright information, infor- 
mation on the distributor and the information on the riqht 
to use. 

[0213] From the shopping brokerage system 9, the in- 
formation on the number of times of access events, the 
information on the concluded agreements and the infor- 
mation on the commodities or services are supplied 
When the shopping brokerage business operator is re- 
sponsible for supplying the program ancillary data 
(metadata), as in the system shown in Fig.1 , the so pre- 
pared program ancillary data is also supplied. 
[021 4] These information are correlated with one an- 
other through the information specifying the respective 
scenes of the respective programs, such as UMID. So, 
the readout or editing may take place freely independ- 
ently of the storage sites of the variegated information. 
Meanwhile, the information selected depending on the 
application is ultimately distributed from the creation 
broadcast system 7 to the reception reproduction sys- 
tem 8. 

[0215] The reception reproduction system 8 acquires 
at least the generic key data and management data, as 
program ancillary data supplied from the creation and 
broadcasting system 7. The data on the commodities or 
services may be distributed along with the program data 
from the creation and broadcasting system 7 or from the 
shopping brokerage system 9 as necessary. 
[0216] On the other hand, the reception reproduction 
system 8 exploits the index picture or the number of 
times of access events, as necessary, to display the da- 
ta pertinent to commodities or services on the display 
picture. The index image may be generated internally 
or obtained from the creation and broadcasting system 
or the reception reproduction system, whilst the number 
of times of access events may be obtained from the cre- 
ation and broadcasting system 7 or from the shopping 
brokerage system. 

[0217] Meanwhile, the layout information for generat- 
ing the display picture for the information concerning the 
commodities or services is routinely furnished from the 
creation and broadcasting system or the shopping bro- 
kerage system. It is of course possible to provide these 
information in an audience side terminal. 
[0218] When the audience requests the detailed in- 
formation as to the commodities or services, the recep- 
tion reproduction system 8 sends the personal informa- 
tion, access information or the management data etc. to 
the shopping brokerage system 9. Similarly, when offer- 
ing to buy the commodities or services appearing on a 



scene of the picture program, the reception reproduction 
system 8 sends the buying offering information, person- 
al information, such as name, address, mail number, 
sex, age etc., access information, for example, the in- 
s formation on the address which holds the information 
and management data etc. to the shopping brokerage 
system 9. 

[0219] This accumulates the program ancillary data 
relevantto thefurnishmentof the information orthepur- 

10 chase of the commodities or services in association with 
the data ancillary to the picture program in the shopping 
brokerage system 9. These information prove beneficial 
information in carrying out commercial transactions by 
the shopping brokerage system 9, while also proving 

is beneficial reference information for the next ensuing 
sponsorship transactions which may be executed on the 
sponsorship transaction brokerage system 6 as de- 
scribed above. 
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(A-2) Illustrative structure 



[0220] Fig.3 shows an illustrative structure of the dig- 
ital broadcast system. Fig.3 shows an illustrative struc- 
ture in case all terminals making up the system are in- 
25 terconnected over a network 10. Meanwhile, Fig.3 
shows an example in which the sponsorship transaction 
brokerage business operator and the shopping broker- 
age business operator are the same or in which different 
undertakers or operators form a virtual LAN (Local Area 
so Network). 

[0221] In this case, the brokerage business operator 
broadcast business operator, sponsor business opera- 
tor and the audience are interconnected overthe broad- 
band WAN (Wide Area network). The connection of the 
35 LAN with the broadband WAN uses a WWW (World 
Wide Web) server 11 operating as a gateway. The 
broadcast business operator, sponsor business opera- 
tor and the audience use their respective dedicated 
WWW servers 11. 
40 [0222] The structure of LAN, supervised and man- 
aged by the brokerage business operator, is now ex- 
plained. This LAN is made up by a system controller 1 2, 
controlling the variegated systems installed on the net- 
work 10, a sponsorship transaction brokerage process- 
's ing management system 13, a shopping brokerage 
processing management system 14, an assessment/ 
settlement processing system 15 and a customer man- 
agement system 16. 

[0223] The sponsorship transaction brokerage 
so processing management system 1 3 is made up of a su- 
pervising section 13A, a retrieval processing computer 
13B, a sponsor operator database 13C, and a WWW 
site creation computer 13D, as main components. In ac- 
tuality, the supervising section 13A of the sponsorship 
ss transaction brokerage processing management system 
is made up by a computer. Meanwhile, the computer is 
made up by a CPU, a storage unit (RAM, ROM etc), a 
peripheral device(s) and an input/output device, inter- 
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connected over a bus. The functions executed by the 
respective computers are routinely implemented as an 
application program (software). 

[0224] The retrieval processing computer 13B re- 
trieves the sponsor operator database 1 3C to retrieve s 
the sponsor operators who are in meeting with the spon- 
sorship request from the broadcast business operator 
(the party who requests the sponsor) or to decide on the 
destination of distributing the sponsorship request. The 
WWW site creation computer 13D is used to prepare a 
display picture for the broadcast business operator as 
the party who requests the sponsorship to input a spon- 
sorship request or to confirm the results of response to 
the sponsorship request. Iri the sponsor operator data- 
base 13C are stored the generic items necessary in de- 
ciding on the destination of distribution of the sponsor- 
ship request and the conditions under which the spon- 
sorship request may be accepted, as registration items. 
[0225] The shopping brokerage processing manage- 
ment system 14 has, as its main components, a super- 
vising section 14A, a meta-pocket forming computer 
1 4B, a meta-mall forming computer 1 4C and a metadata 
database 14D. The supervising section 14A of the shop- 
ping brokerage processing management system is also 
made up by a computer. The computer structure is as 
described above. The functions executed on the respec- 
tive computers are also as described above. 
[0226] The meta-pocket forming computer 14B is 
used for generating the meta-pocket (display for the in- 
formation pertinent to the commodities or services ap- 
pearing in specific scenes or program ancillary data as- 
sociated with the information). The meta-pocket gener- 
ated may be accorded to the broadcasting station or 
suitably distributed in compliance with the audience' re- 
quests. The scene information or the commodity infor- 
mation necessary for meta-pocket generation are rou- 
tinely acquired from the broadcast business operator or 
from the sponsor business operator. Alternatively, it may 
be acquired from the sponsor operator database 1 3C. 
[0227] The meta-mall forming computer 14C is used 
for generating the meta-malls (display image which al- 
lows for overview on one display picture of the meta- 
pockets concerning plural scenes, or the program ancil- 
lary data associated with the information). The so gen- 
erated meta-mall may be accorded to the broadcasting 
station or suitably distributed in compliance with the au- 
dience' requests. The scene information or the com- 
modity information similarly necessary for meta-pocket 
generation are routinely acquired from the broadcast 
business operator or from the sponsor business opera- 
tor. Alternatively, it may be acquired from the sponsor 
operator database 13C. 

[0228] The metadata database 1 4D is used for storing 
the generated meta -pockets or meta-malls. 
[0229] The assessment/settlement processing sys- 
tem 15 is a computer system for executing the assess- 
ment or settlement processing pertinent to the utilization 
of the on-line shopping or service exploitation, such as 



information browsing or downloading chargeable data. 
The customer management system 16 is used for stor- 
ing the information of the customer (audience) handled 
by the assessment/settlement processing system 15 
and for retrieving the information as necessary. 
[0230] With the above-described structure of the sys- 
tem for the brokerage business operators, it is possible 
to realize a digital broadcast system which makes it pos- 
sible to cross-link the scheming stage of the video pro- 
gram (acquisition of sponsor operators) and the stage 
of utilization by the audience of the program ancillary 
data (metadata). 

[0231 ] The foregoing is the explanation of the concep- 
tual structure and an illustrative structure in case of ap- 
plication of the overall system as envisioned in the 
present specification to the digital broadcast system. 
The specified illustrative structure of respective partial 
systems making up the overall system, as envisioned in 
the present specification, is now explained in detail. 

(B) Sponsorship transaction support system 

[0232] As typical of the sponsorship transaction sup- 
port system for on-line realization of the sponsorship 
transaction between a party making a request for spon- 
sorship to a project and registered members, there are 
a system constructed by two parties, namely a party 
making the request and the sponsor business operator, 
and a system constructed by three parties, namely a 
party making the request, a brokerage business opera- 
tor and the sponsor business operator. 
[0233] The former system is realized mainly by ex- 
change via E-mail and exchange through a WWW 
(World Wide Web) picture supervised by the party mak- 
ing the request. The latter system is realized mainly by 
exchange with the E-mail interposed by the brokerage 
business operator and exchange through a WWW 
(World Wide Web) picture supervised by the brokerage 
business operator. 

[0234] Meanwhile, the number of the brokerage busi- 
ness operator need not be singular such that an under- 
taker or operator operating as a wicket of transaction 
may be distinct from an undertaker or operator respon- 
sible for maintenance and supervision of the transaction 
information. It is sufficient in such case if functional ele- 
ments making up the brokerage server as later ex- 
plained are provided in a distributed fashion in the ter- 
minals of different undertakers or operators. 

(B-1) Terminal on the party making a request for 
sponsorship 

[0235] In case of constructing a sponsorship transac- 
tion support system, the terminal of the party making a 
request for sponsorship needs to be provided with a 
function of creating the sponsorship request and trans- 
mitting the so created sponsorship request to the spon- 
sor business operator and a function of receiving a re- 
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sponse from the sponsor business operator to check for 
its content. In actuality, these functions are mostly real- 
ized on one terminal. However, the respective function 
are explained severally for clarifying the technological 
elements required for executing the respective func- 
tions. 



(a) Input device for sponsorship request conditions 
(a-1) Terminal structure 

[0236] Figs.4 and 5 show an embodiment of a spon- 
sorship request condition inputting device having the 
function of inputting the sponsorship request conditions 
Fig.4 shows the case where the sponsorship transaction 
support system is made up of three parties, namely the 
party making a request for sponsorship, the brokerage 
business operator and the registered members. On the 
other hand, Flg.5 shows the case where the sponsor- 
ship transaction support system is made up of two par- 
ties, that is the party making a request for sponsorship 
and the registered members. 

[0237] Reference is first made to Fig.4 showing a de- 
vice for inputting the sponsorship request conditions 20 
made up by a display controller 20A, an input form re- 
cording unit20B, an input data recording unit20C a dis- 
tribution destination specifying unit 20D, a specifying 
rule recording unit 20E, a controller 20F, a communica- 
tion unit 20G, an operating input unit 20H, a bus 20I and 
display means 20J. 

[0238] In Fig.4, the operating input unit 20H and the 
display means 20J are mounted outside of the main 
body unit of the device 20. Alternatively, the operating 
input unit and the display unit may be provided as one 
with the main body unit of the device, as in the case of 
the notebook personal computer, PDA or the portable 
telephone set. Although the recording unit and the com- 
munication unit are shown enclosed in the main body 
unit of the device, these may be mounted outside the 
main body unit. The structure of each functional unit is 
hereinafter explained. 

[0239] The display controller 20A reproduces picture 
data, received or reproduced, in a signal or data form 
suited to the display means 20J. The output signal may 
for example, be a video output (NTSC video, composite 
video or S-video) or an RGB output. The display con- 
troller 20A also has the data decoding function, such as 
MPEG (Moving Picture Experts Group), depending on 
the particular purposes of use. 

[0240] The input form recording unit 20B is a record- 
ing or storage medium for recording an input form for 
sponsorship transaction. Of course, the recording me- 
dium is not to be used exclusively for storage of an input 
form but may also be used for recording or storage of 
other sorts of the information. These mediums may be 
exemplified by, for example, a CD-ROM, mediums of the 
magnetic recording system, such as tape, floppy disc 
hard disc or magnetic card, an optical recording type 



medium, such as write-once CD, DVD or optical card 
magneto-optical recording type medium, such as DVD,' 
MD or MO, and semiconductor memory, such as non- 
volatile memory or RAM. 
s [0241] The input form for recording will be explained 
in detail subsequently. The input form may be pre-fixed 
on the medium or duplicated from other recording me- 
diums. Alternatively, the input form may be downloaded 
through communication networks, such as Internet. The 
™ input form is expanded on a video RAM provided in the 
display controller 20A. 

[0242] The input data recording unit 20C is a record- 
ing medium or a storage medium for recording the data 
input through the operating input unit 20H for communi- 
15 cation purposes. 

[0243] Theinputform recording unit 20B and the input 
data recording unit 20C may be provided within the in- 
side of the display controller 20A. The inputform record- 
mg unit 20B, input data recording unit 20C and the spec- 
ifying rule recording unit 20E may also be mounted out- 
side the casing of the device for inputting the sponsor- 
ship request conditions 20, without being mounted with- 
in the casing. 

[0244] The distribution destination specifying unit 20D 
25 and the specifying rule recording unit 20E are not pro- 
vided for notifying all of the members registered in the 
membership database of the sponsorship request un- 
conditionally, but are provided for executing the 
processing of notifying only the registered members 
30 having to do in some way or directly with the sponsor- 
ship request in question. There may, however, be occa- 
sions where the notification be ultimately made to all 
members. 

[0245] Although the specified functions are provided 
& in Fig.4 on the terminals of the sponsorship requesting 
party, they may also be provided at the terminals of the 
brokerage business operator (brokerage server) It is al- 
so possible to provide only one of the functions in the 
terminal of the sponsorship requesting party and to pro- 
40 vide the otherf unction in the terminal (brokerage server) 
of the brokerage business operator. In sum, the above- 
mentioned functions may be provided in any side termi- 
nal if the distribution destination specifying unit 20D is 
able to read out the specifying rule recorded in the spec- 
ks ifymg rule recording unit 20E to decide on the destina- 
tion of distribution. 

[0246] In the device for inputting the sponsorship re- 
quest conditions 20 having the above-mentioned struc- 
ture, it may be presumed that the distribution of the 
so sponsorship request to the specified registered member 
is routinely performed by the device for inputting the 
sponsorship request conditions itself. Alternatively, data 
stating the destination of distributon may be attached to 
the sponsorship request and transmitted to the broker- 
55 age business operator or to the communication busi- 
ness operator to cause the brokerage business operator 
orthe communication business operatorto send the da- 
ta in multi-destination delivery. 
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[0247] The distribution destination specifying unit 20D 
is the information processing means for implementing 
the function of specifying the registered member(s) hav- 
ing to do in some way or directly with the sponsorship 
request based on the specifying rule stated in the spec- 5 
■tying rule recording unit 20E, as mentioned previously. 
On reading out the input data from the input data record- 
ing unit 20C, the distribution destination specifying unit 
20D executes the membership information having to do 
specifically with the input data. The retrieval processing 
may be executed by the distribution destination speci- 
fying unit 20D directly controlling a membership data- 
base 21 . Alternatively, the distribution destination spec- 
ifying unit 20D may input only the necessary information 
to the membership database 21 to receive only the re- 
trieved results obtained in the membership database 21 . 
[0248] Although Fig.4 shows that the membership da- 
tabase 21 contains desired conditions for distribution 
21 A, previously registered by the registered members, 
it is probable that the membership database 21 lacks in 
such desired conditions for distribution 21 A. Meanwhile, 
if there exist the desired conditions for distribution 21 A, 
the decision as to the area of the distribution is seasoned 
with the desire of the registered members to whom the 
sponsorship request is distributed. The membership da- 
tabase 21 is premised on the presence of the brokerage 
business operator performing the management. 
[0249] By deciding on the area of distribution, based 
on such specific rules, it is possible to save the labor of 
the party making the sponsorship request in specifying 
and inputting the counterparties and inputting the so 
specified counterparties. The operating efficiency may 
be improved because a suitable area becomes the des- 
tination of distribution of the sponsorship on simply filling 
in the input form. Of course, the distribution to all mem- 
bers may be made positively by stating the totality of the 
registered members as the destination of distribution in 
the input item. Conversely, only specified group(s) may 
be stated in the input item as the area of distribution. 
[0250] Since the decision of the area of distribution 
which is based on the specified rule depends on the con- 
tent of registration of the membership database, the 
sponsorship request may be issued to registered mem- 
bers who have not had the opportunities for transac- 
tions. This has the effect of enlarging the business 
chances. 

[0251] The specifying rule may encompass logical 
sum conditions, logical product conditions and other 
complex logical conditions for part or all of the input 
items. Meanwhile, the specific rules may take into ac- 
count the tendency of past distribution or past agree- 
ments, more specifically, data as to whether or not the 
agreements reached conclusion or the number of the 
concluded agreements. The number of access events 
from the consumers to the commodities or services fur- 
nished as the result of the sponsorship event and the 
number of the concluded agreements may also be taken 
into account. If these numbers being more than a preset 



value are taken into the conditions of the logical expres- 
sions, it becomes possible to select those which met 
highly favorble results in the past sponsorship events. 
[0252] The specifying rule recording unit 20 E is a re- 
cording medium for recording a specifying rule. The re- 
cording medium may be a variety of rewritable recording 
mediums, in addition to the read-only memories, such 
as CD-ROM. As aforesaid, the specifying rule recording 
unit 20E may be provided at the terminal of the spon- 
sorship requesting party, or on the terminal of the bro- 
kerage business operator. 

[0253] The specifying rule recorded on the specifying 
rule recording unit 20E may be formed or edited on the 
side sponsorship requesting party. In such case, the in- 
tention of the sponsorship requesting party may be re- 
flected in the area of distribution of the sponsorship re- 
quests. Meanwhile, if the specifying rule is registered, 
the area of the distribution may be the same range at all 
times. Depending on the content of the sponsorship re- 
quest, the sponsorship requests may, of course, be dis- 
tributed to a more desirable area by forming or editing 
the specifying rule in agreement with such content. 
[0254] The specifying rule recorded on the specifying 
rule recording unit 20E may also be formed or edited on 
the side brokerage business operator. In such case, the 
intention of the brokerage business operator may be re- 
flected in the area of distribution of the sponsorship re- 
quest. Since in general the information on the state of 
response to the past distribution area and on the ten- 
dency to agreement conclusion is stored at the broker- 
age business operator, it may be expected to optimize 
the area of distribution. Depending on the content of the 
sponsorship request, the sponsorship requests may 
again be distributed to a more desirable area by forming 
or editing the specifying rule in agreement with such 
content. 

[0255] In any of the above cases, plural sorts of the 
specific rules, having different logical conditions or ref- 
erence items may be provided so that the specific rules 
to be applied may be switched based on the content of 
the sponsorship requests, such as sorts of the projects 
for sponsorship or the targeted customer layers. 
[0256] Of course, the specific rules may include not 
only those initially set by undertakers manufacturing or 
selling sponsorship request condition inputting device 
but also those initially set by a software creating com- 
pany furnishing an application program which permits a 
computer to execute specified conditions as set for the 
destination of distribution. 

[0257] In distinction from the case of unconditionally 
distributing the sponsorship request to all registered 
members, without regard to the specific rules used, the 
area of distribution can be narrowed, in a manner favo- 
rable to the sponsorship requesting party, to the busi- 
ness operator ready to consent to the sponsor, and to 
the business operator which manages the network. For 
example, the area of distribution may be limited to such 
scope of distribution from which sponsorship may be ex- 
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pected, in a manner favorable for the party who has 
made the sponsorship request and who is not desirous 
to distribute the sponsorship request to counterparties 
irrelevant to the content of the sponsorship request 
That is, the party who has made the sponsorship re- 
quest and who is not desirous to distribute the sponsor- 
ship request to counterparties irrelevant to the content 
of the sponsorship request is able to limit the area of 
distribution to such a scope for which the sponsorship 
may be promising. The registered members unwilling to 
consent to the sponsorship are also able with advantage 
to evade the receipt of a large number of the sponsor- 
ship requests the consent to which is against the inten- 
tion of the registered members. The undertaker respon- 
sible for network management may also be exempted 
from the risk of the data flow through the network in- 
creasing to more than a necessary extent thus evading 
the excessive equipment cost. 

[0258J The controller 20F controls various compo- 
nents making up the device for inputting the sponsorship 
request conditions 20, while executing signal process- 
ing operations. In general, the controller is in a micro- 
computer configuration. Usually, the functions of the 
controller 20F are governed by the operation system 
program and by the application program. 
[0259] The communication unit 20G is means for 
communication with outside over a network. For the 
communication unit 20G, a communication equipment 
conforming to the network is used. The network may be 
wired or wireless. For example, the network may be a 
telephone network, an IP network, a dedicated line sys- 
tem, or an ATM network. The communication system 
may be of the line connection type or of the packet con- 
nection type. The network may be of an anlog or digital 
style. The communication unit 20G may be mounted ex- 
ternally, instead of being enclosed in the device, as 
shown in Fig.4. It is through this communication unit 20G 
that the sponsorship request is transmitted. The com- 
munication unit 20G may also be used for communica- 
tion with the membership database 21, as necessary. 
[0260] The operating input unit 20H is used for input- 
ting the content of the sponsorship request or imparting 
various commands to the device for inputting the spon- 
sorship request conditions 20. The operating input unit 
20H may be exemplified by a keyboard, a mouse, a 
pointing device, such as pad, pen or pointer, a dial, in- 
cluding one having a click function, a pressing button 
key etc. The bus 20I is a data inputting/outputting bus 
and may be exemplified by an ISA bus, an EISA bus and 
a PCI bus. 

[0261] The displaying means 20J is used for display- 
ing the information, such as sponsorship request condi- 
tions. Fig.4 shows a case where a device enclosed in a 
casing different from one for the device for inputting the 
sponsorship request conditions 20, for example, a dis- 
play unit of a monitor device or a TV receiver, is con- 
nected as the display means. As aforesaid, thedisplay- 
mg means 20J and the device for inputting the sponsor- 



ship request conditions 20 may be provided in the same 
casing. 

[0262] Meanwhile, the connection system in case of 
providing the respective components making up the de- 
5 vice for inputting the sponsorship request conditions 20 
outside of the main body unit of the device may be wire- 
less, such as with Bluetooth, in place of wired connec- 
tion. 

[0263] The device for inputting the sponsorship re- 
10 quest conditions 20 shown in Fig.4 may be exemplified 
by computer terminals, such as notebook or desktop 
personal computers, PDA, portable telephone sets 
game machines, set top boxes, PC cards, domestic 
electrical utensils, VTR, television receivers, car navi- 
* gation systems, wrist watches having the information 
processing functions, distribution terminals for music or 
games, financial terminals (ATM) etc., in addition to the 
so-called dedicated terminals. 

[0264] The structure of the membership database 21 
20 or the terminal for registered members 22, connected to 
the device for inputting the sponsorship request condi- 
tions 20 over the telephone network or a data network, 
will be explained in detail subsequently. Meanwhile, the 
information registered in the membership database 21 , 
& inclusive of the desired conditions for distribution 21 A,' 
will be explained later in connection with the explanation 
of the sponsorship transaction brokerage server. 
[0265] Reference is now made to Fig.5 showing a de- 
vice for inputting the sponsorship request conditions 20, 
30 which is different from the device shown in Fig.4 in that 
the membership database 21 is provided on the side 
sponsor business operator. Although the membership 
database 21 is provided in the casing of the device for 
inputting the sponsorship request conditions 20, it is not 
35 mandatory that the membership database 21 be en- 
closed in this manner such that it may also be connected 
externally of the casing. The difference of the configu- 
ration of Fig.5 from that of Fig.4 is that the management 
entity of the membership database 21 is the sponsor- 
4 0 ship requesting party. 

[0266] Although the registration of the information 
concerning the sponsor business operator is usually 
made by the sponsorship requesting party, it is also pos- 
sible for a sponsor business operator willing to agree to 
45 the sponsorship request to make individual registration 
over the network. The same applies for a case where 
the sponsor business operator has a desired condition 
for distribution with respect to the distributed sponsor- 
ship request. As in the case of Fig.4, the information reg- 
50 istered in the membership database 21 will be explained 
in detail later in connection with the explanation on the 
brokerage server. 

(a-2) Input form picture 

55 

[0267] Fig.6 shows an example of a sponsorship re- 
quest condition inputting form. Such a favorable effect 
may be expected that, by simply entering the informa- 
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tion in the inputting form, it is possible to create a written 
request (request data) having the necessary minimum 
name information. In the current state of network devel- 
opment of today, this sort of the sponsorship request 
may possibly be made not only in the sponsorship trans- 
action between undertakers or operators, but also be- 
tween operators and individuals and between individu- 
als. So, the necessity is keenly felt for this sort of the 
interfacing technology. 

[0268] An inputting form 25 of Fig. 6 has, as its input 
terms, a project name 25A, time and date of execution 
25 B, project content 25C, a target 25D, desired condi- 
tions for sponsorship 25E to 25G, desired amount 25H 
and input columns 25I for each input item. A transmis- 
sion button 25 J is also indicated on the display picture. 
[0269] Among the items to be entered in the input item 
'project', there are, for example, creation of picture pro- 
grams, including television programs, motion picture 
programs, in turn including those played in a theater, 
and other programs, creation of radio programs, ads, 
business schemes, creation of games, various events, 
including exhibitions, sports, contests, territorial PRs 
and festivals, various funds and various organizations. 
Of course, one or more of these items is entered in cre- 
ating the sponsorship request. 

[0270] As items to be entered in the 'date and time of 
execution', the date and time of project execution and 
broadcast time zones are entered. As items to be en- 
tered in 'content of input items', the project name, such 
as program name, name of a supervisor, objective, such 
as purport or theme of the project, date and time, that is 
date and time of execution or airing, site, such as site 
of execution or airing area, are input. As items to be en- 
tered in 'target (target layer)', territory, age groups, sex, 
revenue, property etc. are entered. As items to be en- 
tered in 'desired conditions for sponsorship', commodi- 
ties or services, quantity, date and time (date and time 
of execution or use), site (site of use of delivery), 
amount, conditions of use, conditions of distribution of 
the sponsorship request (fields of business, specified 
groups, specified persons, specified territories, inclu- 
sive of GPS information) and so forth, are entered. As 
items to be entered in [desired amount], prices for fur- 
nishment of the commodities or services, desired in the 
sponsor, are entered. Alternatively, this item can be en- 
tered as part of the desired conditions for sponsorship. 
[0271] Since these input items represent the crucial 
information in the registered member(s) judging wheth- 
er or not the sponsorship request is to be agreed to, it 
is desirable to use the information optimized in conform- 
ity to the aim in view. In making an entry with a keyboard 
in the respective input columns 25! for respective input 
items, selective entry may be made from respective in- 
put item examples displayed in the respective input 
items. 

[0272] Meanwhile, when a transmission button 25J is 
pressed (clicked) after the entry in the totality of items 
or the basic information, the information recorded in the 



input data recording unit 20C is distributed to the desti- 
nation of distribution decided on by the distribution des- 
tination specifying unit 20D. The technique of deciding 
on the destination of distribution is as described above. 

5 

(a-3) Data of sponsorship request conditions 

[0273] Fig.7 shows a typical structure of data of spon- 
sorship request conditions 26 transmitted from the de- 

10 vice for inputting the sponsorship request conditions 20 
to the terminal for registered members 22. 
[0274] The data of sponsorship request conditions 26 
is made up of the management ID information 26A, the 
client information 26B, transmission destination infor- 
ms mation 26C, client name information 26D, a project 
name 26 E, date and time of execution 26 F, project con- 
tent 26G, target layer information 26H, desired condi- 
tions for sponsorship 26I and other information. 
[0275] The management ID information 26A is the 

20 identification information for the sponsorship client or 
the broadcast business operator to oversee the spon- 
sorship request. The client information 26B is the infor- 
mation for specifying the origin of transmission of the 
sponsorship request on the network, as typified by URL, 

25 E-mail address or telephone number. The transmission 
destination information 26C, which is the information for 
specifying the destination of transmission of the spon- 
sorship request on the network, holds the information 
similar to the client information 26B. Meanwhile, the in- 

30 formation registered in the membership database 21 
and which is pertinent to the registered members deter- 
mined as the destination of distribution by the distribu- 
tion destination specifying unit20D, is used as the trans- 
mission destination information 26C. 

35 [0276] The information registered as the sponsorship 
client is written in the client name information 26D. Al- 
ternatively, the information can be entered in the input- 
ting form of Fig. 6. For the project name 26E, date and 
time of execution 26F and the project content 26G, the 

40 items entered in the inputting form 25 are duplicated and 
used. For the target layer information 26H, desired con- 
ditions for sponsorship 26I, the items entered in the in- 
putting form 25 are duplicated and used. It is noted that 
the desired conditions for sponsorship 26 1 may be as- 

45 sociated with each of the desired conditions for spon- 
sorship, entered with the inputting form 25, or plural de- 
sired conditions for sponsorship may be grouped in one. 

(a-4) Processing operations in making sponsorship 
50 request 

[0277] Fig. 8 shows an inputting processing sequence 
of sponsorship request in the device for inputting the 
sponsorship request conditions 20. When the execution 
55 of the application program for executing the function in 
question is commanded, the controller 20F causes the 
inputting form to be read out from the input form record- 
ing unit 20B (step S1). The application program is stored 
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in a recording medium or a storage medium, not shown 
The controller 20F expands the so read-out inputting 
form on the display controller 20A to output the input 
form in a preset signal format (data format) on the dis- 
play means 20J. As a result, the inputting form is deter- 
mined by the display means 20J on the display picture 
(step S2). 

[0278] The sponsor client, including an operator 
checks the input items displayed, and sequentially in- 
puts the project name, in need of sponsor, date and time 
of execution, sponsorship conditions, including the in- 
formation on the commodities or services desired to be 
requested. During this time interval, the controller 20F 
gives a decision as to whether or not the inputting of the 
necessary minimum information has come to a close 
(step S3). Specifically, the controller verifies whether or 
not the transmission button 25J has been depressed to 
make the decision. That is, when the transmission but- 
ton 25J has been depressed, the controller verifies that 
the inputting has become final. That is, an affirmative 
result is obtained as being the result of decision. If con- 
versely the transmission button 25j has not been de- 
pressed, the controller verifies that the inputting has not 
become final. That is, a negative result is obtained as 
being the result of decision. 

[0279] Alternatively, the decision may be effectuated 
only when the the transmission button 25J has been de- 
pressed. That is, it is verified whether or not the neces- 
sary minimum information has been input, with the ac- 
tuation of the transmission button 25J as a trigger, and 
an affirmative result or a negative result may be' pro- 
duced when the necessary item has been entered or 
when there is any non-entered item, respectively. 
[0280] When the affirmative result is obtained the 
controller 20F affords the information pertinent to spon- 
sorship request to the distribution destination specifying 
unit 20D to execute the processing of identifying the 
destination of distribution of the sponsorship request 
(step S4). When the destination of distribution is decided 
on by this processing, the controller 20F appends the 
management ID information to the information input us- 
ing the inputting form to prepare transmission data (step 
S5). Subsequently, the controller 20F stores the trans- 
mission data prepared and distributes the data through 
the communication unit 20G. By this processing, the 
sponsorship request is distributed to a reasonable area 
of the sponsor business operators (registered mem- 
bers) ready to furnish the commodities or services. 



(b) Response result receiving device 
(b-1) Terminal structure 

[0281] Fig.9 shows an embodiment of a response re- 
sult receiving device having the function of confirming 
the response data notified by the registered member 
conforming to the sponsorship request. Meanwhile, the 
response result receiving device is routinely implement- 



ed on the same terminal as that of the sponsorship re- 
quest condition inputting device. So, the parts or com- 
ponents common to those shown in Figs.4 and 5 are 
depicted by the same reference numerals. Fig.9 shows 
s the case where the cross-sectional view support system 
is mainly made up by three parties; namely a sponsor 
client, a brokerage business operator and registered 
members. 

[0282] The response result receiving device 30 in- 
fo eludes a display control ler20A, a display form recording 
unit 30A, a response data recording unit 30B, a control- 
ler 20F, a communication unit 20G, an operating input 
unit 20H, a bus 20I and display means 20J. Although 
the display means 20J and the operating input unit 20H 
* are provided outside the casing in Fig.9, these may be 
provided within the casing, as the main body unit, as in 
the case of the device for inputting the sponsorship re- 
quest conditions 20. Similarly, the various recording or 
communication units, provided in Fig.9 within the cas- 
20 mg, as the main body unit of the device, may be provided 
outside the casing. 

[0283] The display form recording unit 30A is a re- 
cording medium or storage medium in which to record 
the display form of the response results. It is noted that 
25 the medium may also be used for recording or storage 
of other information, there being no necessity of using 
the medium solely for the display form of the response 
results. The medium may be exemplified by a CD-ROM , 
a magnetic recording medium, such as tape, floppy disc' 
30 hard disc or a magnetic card, an optical recording me- 
dium, such as write-once CD, DVD or optical card, a 
magneto-optical recording medium, such as DVD, MD 
or MO, a semiconductor memory, such as non-volatile 
memory or RAM. 
35 [0284] The response form recorded will be explained 
later in detail. The response form may be pre-fixed on 
the medium, duplicated from other recording mediums 
or downloaded over communication network, such as 
Internet. The response form may be expanded on a vid- 
40 eo RAM provided in e.g., the display controller 20A. 
[0285] The response data recording unit 30B is a re- 
cording or storage medium for recording data input 
through the operating input unit 20H for management 
purposes. Although a RAM is usually employed as this 
45 medium, any sort of the recording medium may be used 
provided that input data can be written therein. There is 
no necessity for the response data recording unit 30B 
to be used exclusively for recording input data. For ex- 
ample, the response data recording unit 30B may be re- 
so alized with the medium which is physically the same as 
the display form recording unit 30A, using a recording 
area distinct from that for the display form recording unit. 
Meanwhile, the display form recording unit 30A and the 
response data recording unit 30B may be provided with- 
55 in the inside of the display controller 20A. 

[0286] Meanwhile, in Fig.9, the membership data- 
base 21 is not provided on the side sponsor client be- 
cause it is here assumed that the sponsorship transac- 
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tion support system is made up mainly by three parties. 
If the sponsorship transaction support system is made 
up mainly by two parties, the membership database 21 . 
is provided on the side sponsorship client. In this case, 
the sponsorship request conditions etc. are recorded in 5 
the final form in the membership database 21 along with 
the management ID information for sponsorship re- 
quest. 

(b-2) Response result confirmation form picture 

[0287] Fig. 1 0 shows an example of a response result 
confirmation form picture. Since the necessary mini- 
mum items in checking the response results to the spon- 
sorship request are indicated in the response result con- 
firmation form picture, the response results can be ex- 
ecuted easily. Moreover, since the display items of the 
response results are demonstrated in the confirmation 
form, the response content can be managed readily. 
That is, for the client receiving many response content, 
it may be easier to compare the results of the response 
content or to grasp the content. In particular, since the 
sponsorship request may be expected to be made not 
only in the sponsorship transaction between different 
operators, but also between the operator and the con- 
sumer and between the individuals, the necessity for the 
above-mentioned interfacing technology is being felt 
keenly. 

[0288] The confirmation form 31 of Fig. 10 has, as its 
display items, the names of responders 31 A, the name 
of the client 31 B, project name 31 C, commodities or 
services that can be offered for sponsorship 31 D, spon- 
sorship conditions 31 E to 31 G, requested amount 31 H 
and display columns 31 1 for the respective display items. 
There is also displayed a confirmed state indicating but- 
ton 31J. 

[0289] The client name 31 B and the project name 31 C 
are the information mainly used by the client in confirm- 
ing the sponsorship request. The names of responders 
31 A and the commodities or services that can be offered 
for sponsorship 31 D are useful in confirming the opera- 
tor who has accepted the sponsorship request and to 
what the response has been made. In the sponsorship 
conditions 31 E to 31 G, the same information as that at 
the time of sponsorship request is usually displayed. 
Should any change be produced in this information, the 
display color is desirably changed by employing a cor- 
responding function in combination. Meanwhile, when 
this function is to be implemented, it is sufficient if the 
sponsorship condition as held is compared to the spon- 
sorship condition as received as to possible coincidence 
to send the result of decision to the display controller 
20A or to the controller 20F. 

[0290] In the amount requested 31 H, the price re- 
quested by the business operator agreeing to the spon- 
sor is displayed. Meanwhile, if this item only can be input 
by the registered members, condition comparison may 
be made easily in case plural responses are made to 



the same sponsorship request. When the sponsorship 
transaction is to be validated for the content of the above 
display item, it suffices to depress the confirmed state 
indicating button 31J. 

(b-3) Response result data 

[0291] Fig. 11 shows an illustrative structure of re- 
sponse result data 32 transmitted from the terminal for 
registered members 22 to the response result receiving 
device 30. 

[0292] The response result data 32 is made up by the 
management ID information (responder side) 32A, the 
management ID information (client side) 32B, a re- 
sponse source information 32C, a transmission destina- 
tion information 32D, the name of the registered mem- 
bers 32 E, content of commodities or services that can 
be offered for sponsorship 32F, sponsorship condition 
information 32G and other information. 
[0293] The management ID information (responder 
side) 32A, appended by the registered member accept- 
ing the sponsorship request, is the identification infor- 
mation used not only by the registered member but also 
by the brokerage business operator or by the client re- 
ceiving the response result for supervising the response 
result data. On the other hand, the management ID in- 
formation (client side) 32B is the identification informa- 
tion used by e.g., the client receiving the response re- 
sults in supervising the sponsorship request. 
[0294] The response source information 32C, speci- 
fying the transmission source of the response result on 
the network, may be exemplified by URL, E-mail ad- 
dress or the telephone number. The transmission des- 
tination information 32D specifies the destination of 
transmission of the sponsorship request on the network. 
The information of the same sort as the response source 
information 32C is held in the transmission destination 
information. Meanwhile, the information stated in the da- 
ta of the sponsorship request conditions 26 is used as 
the transmission destination information 32D. 
[0295] In the name of the registered members 32E, 
the names of the registered members accepting the 
sponsorship request are entered. This information facil- 
itates identification of the business operator accepting 
the sponsorship request. By recording the information 
as separate items, data re-arraying or extraction can be 
carried out readily. An entry into the content of commod- 
ities or services that can be offered for sponsorship 32F 
is made to permit the facilitated confirmation of the com- 
modities or services with which the business operator 
transmitting the response result data agreed to the 
sponsor. The information corresponding to the desired 
conditions for sponsorship 26! is duplicated and entered 
in the sponsorship condition information 32G. It is noted 
that, if the registered member accepting the sponsor- 
ship request has made separate correction of the spon- 
sorship conditions, the as-corrected information is re- 
corded. In this case, the information indicating the fact 
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of the correction made is desirably stored in combina 
tion. 



resulte' 006881 " 9 ° perations on rec eiptof the response 

£0296] On confirming the receipt of the response re- 
sult data, the controller 20F commands the execution of 
the application program for executing the function in 
question to cause the display form to be read out from 
the display form recording unit 30A. In this case, the ap- 
plication program is again stored in the recording or stor- 
age medium, not shown. On receipt of the response re- 
sult data, the controller 20F causes the received re- 
sponse result data to be recorded in the response data 
recording unit 30B. 

[0297] The controller 20F causes the response result 
data and the display form to be read out and routed to 
the display controller 20A to expand the information so 
read out in a display RAM provided in the display con- 
troller 20A. In case of non-coincidence of the read-in re- 
sponse result data with the item of the display form the 
controller 20F specifies a site in which to embed the re- 
sponse result data. The Information expanded on the 
display controller20A is outputto the display means 20J 
in a preset signal format (data format). As a result the 
content of the response result data 32 is displayed as 
an overview on the display picture. If there are plural 
response result data 32, these may be displayed se- 
quentially or simultaneously on one display picture 
[0298] When the content of the response result data 
are confirmed and the business operator who is to val- 
idate the sponsorship transaction is determined the 
confirmed state indicating button 31J is operated on the 
display picture for the response result data received 
from the business operator. When a button for validating 
the sponsorship transaction is provided separately from 
the confirmed state indicating button 31J, the former is 
pressed. At any case, if the operation is finally deter- 
mined by button actuation, the finally determined infor- 
mation is recorded on a recording medium, not shown 
or in the response data recording unit 30B, whilst the 
fact of validation is registered in the membership data- 
base 21 . 

[0299] The fact of validation of the transaction is no- 
tified, either directly or indirectly, to the registered mem- 
ber for whom the sponsorship transaction has become 
valid. Preferably, the fact of non-validation of the spon- 
sorship transaction is notified to the registered member 
(s) for whom the sponsorship transaction has not be- 
come valid. 

(B-2)Terminal of thesponsorship transaction brokerage 
business operator 

(a-1) Terminal structure 

[0300] Fig. 1 2 illustrates an embodiment of a broker- 



age transaction brokerage server acting as a broker for 
sponsorship transaction. Meanwhile, the brokerage 
transaction brokerage server is used in case the spon- 
sorship transaction support system is constructed by 
three parties, namely the sponsor client, brokerage 
business operator and registered members 
[0301 ] it is noted that the membership database 21 in 
Fig.12 is the same as the membership database 21 
used in the explanation on the device for inputting the 
10 sponsorship request conditions 20. In the case of Fig 
12, the membership database 21 is provided within the 
casing of a sponsorship transaction brokerage server 
35. However, it is naturally not mandatory to provide the 
membership database in the same casing as that of the 
main body unit. For example, the membership database 
may be provided on the network. It is sufficient if a sole 
system is virtually constructed over a network The 
same holds for other functional units. The network may 
be wireless, instead of being a cable network. 
[0302] The sponsorship transaction brokerage server 
35, shown in Fig.12, is made up by a controller 35A a 
specifying rule recording unit 35B, a distribution desti- 
nation specifying unit 35C, a distributing section 35D a 
sponsor pre-registration accepting unit 35E, a pre-reg- 
istration condition decision unit 35F, a communication 
unit 35G and a bus 35H. Meanwhile, this construction 
is used when the brokerage business operator decides 
on the destination of distribution of the sponsorship re- 
quest. If the decision on the destination of distribution 
of the sponsorship request is to be made at the client 
terminal, the sponsorship transaction brokerage server 
is made up by the controller 35A, membership database 
21 and the communication unit 35G. The construction 
of various components is hereinafter explained 
35 [0303] The controller 35A represents means for con- 
trolling various components making up the sponsorship 
transaction brokerage server 35. In general, the control- 
ler is of the micro-computer configuration. The functions 
of the controller 35A are usually prescribed by the op- 
eration system program and by the application program 
In the present embodiment, the sponsorship transaction 
brokerage server 35 also executes the processing of 
registenng the responses to the sponsorship request 
sent from the registered members in the membership 
database 21 and the processing of accepting the spon- 
sor client. 

[0304] Meanwhile, a dedicated processor for execut- 
ing the functions, that is, a response result data receiver 
for receiving the response to the sponsorship request 
a response result data registration unit for registering 
the response result data in the membership database 
21 and a response result data transmitter for sending 
the response result data to a destination of transmission 
entered in the data, may be used. It is to be noted that 
these functions are implemented as the functions of the 
controller 35A. Additionally, the server also performs the 
processing operation of registering the information on 
the finally determined sponsorship transaction in the 
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membership database 21 . 

[0305] The specifying rule recording unit 35B is equiv- 
alent to the specifying rule recording unit 20E of the de- 
vice for inputting the sponsorship request conditions 20 
(Fig.4). On the other hand, the distribution destination s 
specifying unit 35C is equivalent to the distribution des- 
tination specifying unit 20D of the device for inputting 
the sponsorship request conditions 20 (Fig.4). Conse- 
quently, the foregoing explanation on the specifying rule 
recording unit 20 E and the distribution destination spec- 
ifying unit 20D is valid for the specifying rule recording 
unit 35B and the distribution destination specifying unit 
35C, respectively. 

[0306] The specifying rule recorded in the specifying 
rule recording unit 35B is assumed to be that created or 
edited by the brokerage business operator. In this case, 
the intention of the brokerage business operator can be 
reflected in the area of distribution of the sponsorship 
request. By and large, the information concerning the 
state of response and the tendency to agreement con- 
clusion for the past area of distribution is stored in the 
brokerage business operator, and hence the area of dis- 
tribution may be expected to be optimized. It is noted 
that the specifying rule may be that created or edited on 
the part of the sponsorship client, in which case the cli- 
ent's intention may be reflected in the area of distribution 
of the sponsorship request. 

[0307] The distribution destination specifying unit 35C 
retrieves the membership database 21 based on the 
specifying rule recorded in the specifying rule recording 
unit 35B to implement the function of identifying the reg- 
istered member having to do in someway or directly with 
the sponsorship request. In this case, the membership 
database 21 may be retrieved by the distribution desti- 
nation specifying unit 35C directly controlling the mem- 
bership database 21 , or by sending only the necessary 
information to the membership database 21 to receive 
only the retrieved results. 

[0308] Meanwhile, in the case of Fig. 12, the pre-reg- 
istration condition decision unit 35 F is present in the 
sponsorship transaction brokerage server 35, such that 
the decision processing needs to be executed prior to 
the processing of specifying the destination of distribu- 
tion. So, the operation on receipt of the sponsorship re- 
quest partially differs from the foregoing explanation. 
The operation on receipt of the sponsorship request will 
be explained subsequently. 

[0309] The sponsor pre- registration accepting unit 
35 E and the p re-registration condition decision unit 35F 
represent means operating so that, when the intention 
of the business operator requesting the sponsor is in 
keeping with that of the responding business operator, 
the transaction with the two business operators is pre- 
ferred to transactions between other business opera- 
tors. By the way, the processing of specifying the desti- 
nation of distribution based on the specifying rule rep- 
resents means for optimizing the area of distribution of 
the sponsorship request. The sponsor pre-registration 



accepting unit 35 E and the pre-registration condition de- 
cision unit 35F may be provided in the device for input- 
ting the sponsorship request conditions 20. 
[0310] The sponsor pre-registration accepting unit 
35 E represents means for pre-registering, in the mem- 
bership database 21 , the conditions desired when the 
registered member willing to accept the sponsorship re- 
quest accepts the sponsorship request. By and large, 
the content of the registration need not be identical with 
but may be completely equated to the sponsorship re- 
quest conditions input by the device for inputting the 
sponsorship request conditions 20. The information for 
registration will be explained subsequently. 
[0311] The pre-registration condition decision unit 
35F is information processing means for implementing 
the function of verifying or retrieving, on receipt of the 
sponsorship request, and prior to decision on the desti- 
nation of distribution of the sponsorship request, wheth- 
er or not there is any registered member who has reg- 
istered the condition satisfying the sponsorship request 
in question. The pre-registration condition decision unit 
35F is configured for retrieving the membership data- 
base 21 based on the sponsorship request received 
through the communication unit 35G to verify the pos- 
sible presence of the pre-registration information which 
accords with the conditions of the sponsorship request. 
[031 2] The decision here is basically the coincidence 
detection. As for sponsorship conditions that can be ex- 
pressed quantitatively, for example, the amount re- 
quested or quantity, the pre-registration information in 
meeting with the sponsorship request conditions is 
deemed to be present in case the conditions are more 
favorable to the client than the sponsorship condition, 
for example, in case the amount requested is lower than 
the amount stated in the sponsorship request, or in case 
the quantity is larger than that stated in the sponsorship 
request. If the number of the requirements of the spon- 
sorship request received is larger than that of the spon- 
sorship condition registered, a decision is given that the 
sponsorship request conditions are met when the re- 
quirements of the sponsorship request are met as con- 
cerns the registered requirements. 
[031 3] When there is any registered member meeting 
the sponsorship request, the pre-registration condition 
decision unit 35 F notifies both the registered member in 
question and the sponsorship client that there are the 
counterparties who are in meeting with the two without 
executing the processing of identifying the destination 
of distribution. In this case, the conditions requested by 
the counterparties are notified. If the number of the reg- 
istered member satisfying the sponsorship request is 
singular, it is theoretically possible to assume that the 
transaction has been concluded at this time point. How- 
ever, since this is the result of mechanical detection of 
coincidence, the sponsorship transaction is desirably 
concluded subject to agreement between the two par- 
ties advised of the fact of the coincidence detection. 
Conversely, when there is no registered member satis- 
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tying the sponsorship request, the pre-registration con- 
dition decision unit 35F operates for sending the re- 
ceived sponsorship request to the distribution destina- 
tion specifying unit 35C. 

[0314] Similarly to the control unit 20G in the device 
for inputting the sponsorship request conditions 20 the 
communication unit 35G represents means for commu- 
nication with outside over the network. So, a communi- 
cation equipment conforming to the network is used as 
the communication unit 35G. The types of the networks 
usable are as described above. The communication unit 
35G may be mounted externally, instead of being en- 
closed, as shown in Fig.12. The sponsorship request 
and the response from the registered member etc. are 
transmitted overthis communication unit 35G The com- 
munication unit may also be used for communication 
with the membership database 21 , as necessary. 
[0315] The bus 35H is a data inpuVoutput bus and 
may be exemplified by, for example, an ISA bus, a EISA 
bus and a PCI bus. Although the operating input unit or 
the display means are not shown in Fig.12, there are 
provided components equivalent to the operating input 
unit 20H and the display means 20J of Fig.4. 
[0316] The membership database 21 and the desired 
conditions for distribution 21 A will be explained sepa- 
rately later in detail. Meanwhile, the sponsorship trans- 
action brokerage server 35, shown in Fig.12 may be 
exemplified by computer terminals, such as notebook 
or desktop personal computers, PDA, portable tele- 
phone sets, game machines, set top boxes, PC cards 
domestic electrical utensils, VTR, television receivers' 
car navigation systems, wrist watches having the infor- 
mation processing functions, distribution terminals for 
music or games, financial terminals (ATM) etc., in addi- 
tion to the so-called dedicated terminals. 



(a-2) membership database 

[0317] Fig.13 shows an illustrative data structure of 
the membership database 21. The membership data- 
base 21 is made up of storage means, comprised of a 
large capacity recording or storage medium, a commu- 
nication unitfor having communication with outside and 
a controller supervising the entire system, such as a 
computer system comprised of a micro-processor 
RAM, ROM, and an input/output unit. The large-capacity 
recording medium may be exemplified by a magnetic 
disc, such as a hard disc, a magneto-optical disc, such 
as MO, an optical disc, such as DVD and a semiconduc- 
tor memory. 

[0318] The membership database 21 is made up of 
desired conditions for distribution 21 A, management in- 
fonmation 21 B to 21 L, goods-related information 21 M to 
21 R, service-related information 21 Q to 21 S, conditions 
under which the sponsorship request 21T is responded 
to, and other information. There may be a system in 
which the desired conditions for distribution 21 A are not 
registered. In a system adapted for registering the de- 



sired conditions for distribution 21 A, it is up to discretion 
of the sponsor business operator whether or not the in- 
formation in question is to be registered. 
[0319] In the following explanation, such a system is 
5 presupposed which registers the desired conditions for 
distribution 21 A. The desired conditions for distribution 
21 A represent the information corresponding to the con- 
tent of the sponsorship request distributed and which 
are pre-registered by the registered members (sponsor 
10 business operators) as the recipients of the sponsorship 
request. For example, the information on the commod- 
ities or services that can be offered for sponsorship the 
information specifying the sponsorship client, time (time 
of furnishment or execution etc.) or place (place of fur- 
mshment or execution, place of residence of the coun- 
terparty etc.) is registered. 

[0320] The desired conditions for distribution 21 A op- 
erate as the requirements for further narrowing the area 
of distribution as specified by the general registration 
information (such as management information, goods- 
related information or the service-related information) 
rather than as specified sponsor accepting require- 
ments pertinent to the individual commodities orservic- 
es as under the conditions 21T under which the spon- 
sorship request is accepted. Meanwhile, the general 
registration information is pre-registered by the regis- 
tered members or by the brokerage business operator. 
[0321] The management information may be exem- 
plified by the management ID information 21 B, registra- 
*> tion date information 21 C, updating date information 
21 D, registrant name information 21 E, business sort in- 
formation 21 F, place of residence information 21 G tel- 
ephone number 21 H as the contact site information 
mail address information 21 1, URL information 21 J GPS 
35 information 21 K, mail number information 21 L etc. 
[0322] The management ID information 21 B is the in- 
formation for specifying a sequence of the information 
The registration date information 21 C and the updating 
date information 21 D represent the data management 
information. The registrant name information 21 E is the 
name of the registered member as an entity of the spon- 
sorship transaction. The business sort information 21 F 
is the recorded information concerning the main busi- 
ness sorts of the registered members. This information 
operates as an index in specifying the area of distribu- 
tion of the sponsorship request. In general, it is the reg- 
istered member that makes this registration. However, 
the brokerage business operator sometimes makes this 
registration. The place of residence information 21 G 
generally records the information concerning the place 
of residence of the main office and branch offices to 
which belong staff members in charge of the processing 
of the sponsorship transaction. This information also op- 
erates as an index in specifying the area of distribution 
55 of the sponsorship request. 

[0323] The telephone number 21 H is the information 
used by the brokerage business operator and the spon- 
sorship request in making a telephone call. The mail ad- 
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dress 21 1 is used as the address information for making 
sponsorship request or the relevant communication 
over the network. The URL information 21 J is the infor- 
mation for the registered member to access the infor- 
mation laid open by the registered members. The GPS 5 
information 21 K is used for displaying the information 
pertinent to the place of residence or the site of furnish- 
ment on an electronic map. This information may also 
be used as an index in specifying the area of distribution 
of the sponsorship request. The information may be reg- 10 
istered as a site of destination in the car navigation sys- 
tem to visit the counterparty. The mail number 21 L is 
also effective as the information used for narrowing the 
site of residence etc. 

[0324] The goods-related information and the serv- 1$ 
ice-related information are individually registered for 
what the registered member is willing to supply or the 
goods handled by the registered member. That is, when 
there are plural goods or services to be registered, a 
number of sets of the information corresponding to the 20 
number of the commodities or services in question are 
recorded. As the goods- related information, the infor- 
mation on the site of furnishment of the commodities or 
services 2 1 M, the information on the classification of the 
commodities handled 21 N, the information on the 25 
names of the commodities handled 21 0 or the informa- 
tion on the names of producers 21 P etc. are recorded. 
As the service related information, the information on 
the site of furnishment 21 Q, the information on the code 
of services handled 21 R and the information 21 S on the 30 
names of services handled, are recorded. 
[0325] As the site information for furnishing the com- 
modities or services, the information associated with in- 
dividual commodities or services is recorded. The infor- 
mation on the classification of the commodities handled 35 
21 N or the information on the code of services handled 
21 R represents the information beneficial to processing 
on the computer and is desirably appended on the basis 
of a unified standard, such as JIS. As the information on 
the code of services handled 21 R and the information 40 
21 S on the names of services handled, the trademarks 
used by the registered members furnishing the com- 
modities or services, for example, are recorded. How- 
ever, the generic appellation of the commodities or serv- 
ices may also be used. The name of the producer 21 R 45 
is the information on the producer. Although the infor- 
mation associated with services is not registered, the 
name of the party directly furnishing services is recorded 
in case the registered member is the party acting as in- 
termediary for the services. so 
[0326] The conditions under which the sponsorship 
request is accepted 21 T represent the information for 
the registered member to register specified conditions 
in case the registered member accepts the sponsorship 
request relevant to specified commodities or services, ss 
This information is the information registered by the 
sponsor p re-registration accepting unit 35E and refer- 
enced by the pre-registration condition decision unit 35F 



who has received the sponsorship transaction. As afore- 
said, the conditions under which the sponsorship re- 
quest is accepted 21 T are usually registered for items 
similar to the input items of the sponsorship request. 

(a-3) Processing operation 

[0327] The content of the brokerage operations exe- 
cuted by the sponsorship transaction brokerage server 
is hereinafter explained for respective cases. In the fol- 
lowing explanation, the sponsorship transaction broker- 
age server 35 is assumed to have the structure shown 
in Fig.12. 

(1 ) On pre-registration of the sponsorship conditions 

[0328] On receipt from the terminal for registered 
members 22 of a notice having the conditions under 
which to accept the sponsorship request as data con- 
tent, the controller 35A sends the notice to the sponsor 
pre-registration accepting unit 35E. The sponsor pre- 
registration accepting unit 35 E retrieves the member- 
ship database 21 , with the information of the registered 
member who issued the notice as the clue to add the 
conditions under which to accept the sponsorship re- 
quest 21 T as the registration information of the relevant 
registered member. This addition of the information is 
carried out for each individual commodity and service. 
The information may also be individually deleted on no- 
tification from the registered member. By employing the 
information, the registered member is able to set the 
condition adapted to the market status. It is however up 
to the registered member whether the information is to 
be used for short-term transaction purposes or for long- 
term purposes. 

(2) On receipt of sponsorship request 

[0329] Fig. 14 shows the processing sequence exe- 
cuted in case the sponsorship transaction brokerage 
server 35 receives the sponsorship transaction. On as- 
certaining the receipt of the sponsorship request from 
the sponsorship client, the controller 35A captures the 
sponsorship request conditions of the sponsorship re- 
quest in a storage area, such as RAM, not shown, while 
notifying the receipt of the sponsorship request to the 
pre-registration condition decision unit 35F (step S11). 
[0330] The pre-registration condition decision unit 
35F verifies whether or not there are those conditions 
in meeting with the content of the sponsorship request 
among the conditions 21T which are registered in the 
membership database 21 and under which the sponsor- 
ship request is accepted (step S12). If there is one or 
more p re-registered information that is in meeting with 
the sponsorship request, the affirmative results are ob- 
tained, so that the processing transfers to step S13. If 
there is no pre-registered information that is in meeting 
with the sponsorship request, a negative result is ob- 
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tamed, so that processing transfers to step S14. It is not- 
ed that the negative result is obtained in case the con- 
ditions which are in meeting with the sponsorship re- 
quest 21 T are registered for none of the registered 
members. 

[0331] When the processing has transferred to step 
S13, the pre-registration condition decision unit35F no- 
tifies to both the registered member who has pre-regis- 
tered the conditions which are in meeting with the spon- 
sorship request and the client who issued the sponsor- 
ship request that the counterparties are presentwho are 
able to meet the conditions of the opposite side parties 
This notice is made through the communication unit 
35G. The processing operations in case the sponsor- 
ship transaction is finally determined for the notification 
will be explained subsequently. 

[0332] When the processing transfers to step S1 4 the 
processing operation is transferred from the pre-regis- 
tration condition decision unit35F to the distribution des- 
tination specifying unit 35C. The processing operations 
in the distribution destination specifying unit 35C, now 
explained, similarly apply in case the function is provid- 
ed at a terminal of the sponsorship client. The distribu- 
tion destination specifying unit 35C reads out the spec- 
ifying rule used for specifying the destination of distribu- 
tion from the specifying rule recording unit 35B to re- 
trieve the membership database 21 based on the so 
read-out specifying rule. The distribution destination 
specifying unit 35C then executes retrieval on the ge- 
neric registration information (management information 
goods-related information and service-related informa- 
tion) from among the information registered in the mem- 
bership database 21 . 

[0333] So, the distribution destination specifying unit 
35C finally determines the registered members to whom 
the sponsorship request is to be distributed (step S16). 
Usually, the registered members in meeting with the 
specifying rule are extracted as the destination of distri- 
bution of the sponsorship request. If there Is no regis- 
tered member in meeting with the specifying rule, the 
totality of the registered members registered in' the 
membership database 21 are finally determined as the 
destination of distribution of the sponsorship request. 
[0334] The distribution destination specifying unit 35C 
verifies whether or not the desired conditions for distri- 
bution 21 A have been registered for the registered 
members finally determined as the destination of distri- 
bution (step S16). When the affirmative result is ob- 
tained, that is when there exist the desired conditions 
for distribution 21 A, the distribution destination specify- 
ing unit 35C verifies whether or not the sponsorship re- 
quest also meets the desired conditions for distribution 
21 A registered by the respective registered members 
The distribution destination specifying unit extracts only 
the members in meeting with the desired conditions for 
distribution 21 A (step S17). If conversely the negative 
result is obtained, that is if there are no desired condi- 
tions for distribution 21A, the distribution destination 



specifying unit 35C skips the step S17. That is, since 
there is no reason for not distributing the sponsorship 
request to the registered member(s) who satisfy the 
above condition, the registered member(s) is finally de- 
s termined at this stage to be comprised within the area 
of distribution. 

[0335] When the ultimate area of distribution is finally 
determined, the distribution destination specifying unit 
35C distributes the sponsorship request to the finally de- 
io termined registered member(s) (step S1 8). At this time 
the distribution destination specifying unit 35C notifies 
the presence of the information using the mail address 
information 21 1 registered in the membership database 
21 . It is noted that, if the fax number is registered, call 
is may be made by fax. The above completes the process- 
ing of distributing the sponsorship request. 

(3) On receipt of the response result data 

20 [0336] On receipt of a notice having as data content 
the reply result for sponsorship request from the regis- 
tered member who has received the sponsorship re- 
quest, the controller 35A causes the content of the no- 
tice to be registered in the membership database 21 . It 
25 is noted that there are two responses, namely the re- 
sponse agreeing to the sponsorship and the response 
not agreeing to the sponsorship. This registered infor- 
mation may be used in specifying the destination of dis- 
tribution for subsequent sponsorship requests. For ex- 
30 ample, if the response results to similar sponsorship re- 
quest indicate non-agreement to the sponsorship, the 
registered member(s) who made the response may be 
omitted from the group of the succeeding destination of 
distribution. 

35 [0337] Since it may be probable that the previous 
sponsorship was only fortuitously not agreed to, such 
omission is desirably made only in such case where 
there is annexed the information to the effect that distri- 
bution of the sponsorship request of the same sort is not 
40 henceforth desired by the registered members, in order 
to make much of the goodwill on the part of the regis- 
tered members. 

[0338] The controller 35A selectively sends the affirm- 
ative response results to the client. The controller 35A 
« at this time transmits data based on the information on 
the destination of transmission annexed to the response 
result data 32 or the liaison site information of the spon- 
sorship request as specified from the management ID 
information. It is noted that, at this time point, plural re- 
50 sponse result data 32 are transmitted to the sponsorship 
request. 

(4) On receipt of the notification of registered results 



55 [0339] On receipt from one or both of the terminal of 
the sponsorship request 20 and the terminal for regis- 
tered members 22, indicating the conclusion of the final- 
ly determined sponsorship transaction, as data content, 
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the controller 35A registers the content of the notification 
in the membership database 21 . 
[0340] This registration information is used for speci- 
fying the area of distribution of the next and following 
sponsorship transactions or as the management infor- s 
mation for the project as the object of sponsor. For ex- 
ample, when the subject-matter of the sponsorship is 
the creation of the picture program, the registered infor- 
mation may be furnished as data ancillary to the pro- 
gram to the program producer. Since the data can be 
extracted, the utilization efficiency of the information in 
question may be improved. That is, such a wasteful in- 
cidence may be evaded in which data of the same con- 
tent as viewed from the entire system is input a number 
of times at different time points or places. 
[0341] Thus, the membership database 21 not only is 
valuable for sponsorship transaction but also serves as 
a beneficial information source in business transactions 
and for various projects. Meanwhile, if the data readout 
from the membership database 21 is chargeable, it is 
possible to collect the fee needed in the inputting or 
management. 

[0342] Meanwhile, as the data fetching method from 
the membership database 21 , transmission paths, wired 
or wireless, may be used, or a recording medium may 
be used. The recording medium may be enumerated by, 
for example, magnetic recording type medium, such as 
tapes, floppy discs, hard discs or magnetic cards, optical 
recording mediums, such as write-once CD, DVD or op- 
tical cards, mediums of the magneto-optical disc type, 
such as DVD, MDorMO, and semiconductor memories. 

(B-3) Registered member's terminal 

[0343] In the case of constructing the sponsorship 
transaction support system, the functions directly rele- 
vant to the sponsorship transaction, such as the func- 
tions of receiving the sponsorship request and transmit- 
ting the response of the registered members to the re- 
ceived sponsorship request, and the function for regis- 
tration necessary for receiving the distributed sponsor- 
ship request, are required of the terminal of the regis- 
tered member. In actuality, these functions are imple- 
mented in a majority of cases in one terminal. The fol- 
lowing explanation is, however, made individually for 
clarifying the technological elements necessary for ex- 
ecuting respective functions. 

(a) Device for receiving the sponsorship request 

(a-1) Terminal structure 

[0344] Fig. 1 5 illustrates an embodiment of a sponsor- 
ship request receiving device 22 having the function of 
receiving the sponsorship request and the function of 
accepting the request. The sponsorship request receiv- 
ing device 22 includes a display controller 22A, a re- 
quest content confirmation form recording unit 22B, a 



sponsorship request recording unit 22C, a controller 
22D, a communication unit 22E, an operating inputting 
unit 22F, a bus 22G and display means 22H. 
[0345] In Fig. 15, the display means 22H and the op- 
erating inputting unit 22 F are shown connected outside 
the main body unit of the device. However, the operating 
inputting unit and the display means may also be con- 
nected as one to the main body unit of the device, as in 
the case of a notebook personal computer, a PDA and 
a portable telephone set. Although the recording unit 
and the communication unit are shown enclosed in the 
main body unit, these may also be connected to outside 
the main body unit. In the case of the external connec- 
tion, connection may be wired or wireless. The struc- 
tures of the respective functional units are hereinafter 
explained. 

[0346] Similarly to the display controller 20A, the dis- 
play controller 22A is means for reproducing and out- 
putting the received or reproduced video data in a signal 
format (data format) complying with the display means 
22H. As forthe signal format and the decoding functions 
of the display controller 22A, what has been said of the 
display controller 20A applies. 

[0347] The request content confirmation form record- 
ing un it 22B is a recording or storage medium for record- 
ing the confirmation form used for displaying the content 
of the sponsorship request. However, the medium may 
be used simultaneously for recording or storing other in- 
formation without being used exclusively as an inputting 
form for sponsorship transaction. The medium may, for 
example, be a CD-ROM, a magnetic recording type me- 
dium, such as tape, floppy disc, hard disc or a magnetic 
card, an optical recording medium, such as write-once 
CD, DVD or optical card, magneto-optical recording 
type medium, such as DVD, MD or MO, and semicon- 
ductor memory, such as non-volatile memory or RAM. 
[0348] The inputting form recorded will be explained 
later in detail. The inputting form may be pre-fixed on 
the medium, duplicated from other recording mediums 
or downloaded over a communication network, such as 
Internet. The inputting form may be expanded on a video 
RAM provided in e.g., the display controller 20A. 
[0349] The sponsorship request recording unit 22C is 
a recording or storage medium for recording the spon- 
sorship request condition data 26 received from the 
communication unit 22E. Meanwhile, the sponsorship 
request condition data and other information exchanged 
in connection with the sponsorship transaction may be 
data composed only of data shown in Fig. 7, data stating 
the layout information (style sheet), as is the HTML (Hy- 
pertext Markup Language) file, or the XML (Extensible 
markup Language) file, in which the layout information 
(style sheet) is isolated from data. 
[0350] The controller 22D is means for controlling var- 
ious components making up the sponsorship request re- 
ceiving device 22 and for executing signal processing, 
and is generally of the micro-computer configuration. 
The functions of the controller 22 F are usually pre- 
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scribed by the operation system program and by the ap- 
plication program. 

[0351] Similarly to the communication unit 20G of Fig 
4, thecommunication unrt22E is means for having com- 
munication with outside over a network. The network to 
be used is as described above with reference to Fig 4 
The operating inputting unit 22F is similarto the operat- 
ing inputting unit 20H of Fig.4, while the display means 
22H is similar to the display means 20J of Fig.4. 
[0352] The sponsorship request receiving device 22 
may be formed by computer terminals, such as note- 
book or desktop personal computers, PDA, portable tel- 
ephone sets, game machines, set top boxes, PC cards 
domestic electrical utensils, VTR, television receivers' 
car navigation systems, wrist watches having the infor- 
mation processing functions, distribution terminals for 
music or games, financial terminals (ATM) etc., in addi- 
tion to the so-called dedicated terminals. 

(a-2) Confirmation form picture 

[0353] Fig.1 6 shows an example of a sponsor content 
confirmation form. If this confirmation form is used the 
necessary minimum items in verifying whether or not the 
sponsorship transaction is to be made may be present- 
ed in accordance with preset rule, so that the sponsor 
content can be confirmed and managed extremely read- 
ily. In particular, if the registered member receives a 
large quantity of sponsorship requests, it becomes pos- 
sible to standardize the judgment. Moreover, as a result 
of the standardized items of confirmation, there may be 
provided a system convenient to use not only in spon- 
sorship transaction among different operators, but also 
in sponsorship transaction between operators and indi- 
viduals and between individuals without regard to the 
age groups or sex of the users. 
[0354] A confirmation form 36 shown in Fig 16 in- 
cludes, as display items, a client 36A, a project name 
36B, date and time of execution 36C, project content 
36D, a target 36E, desired sponsorship conditions 36F 
to 36H, desired amounts 36I and detailed information 
columns 36I corresponding to the display items. In the 
display picture, there are indicated a button 36K oper- 
ated when the sponsorship request is accepted, that is 
not agreed to, and a button 36L operated when the 
sponsorship request is not accepted, that is not agreed 
to. 

[0355] In the columns 36I of the detailed information 
associated with respective display items, the corre- 
sponding information in the sponsorship request condi- 
tion data 26 received as the sponsorship request is du- 
plicated and displayed. For example, in the case of Fig 
16, it may be seen that the client is [A-l-U Broadcasting 
Company], with the project name being [Television Pro- 
gram ABCD]. It may also be seen that the date and time 
of execution is [aired as from 2001, January 3, 21:00 
until 22:00], with the project name being [what is likely 
to be in vogue] and the target being [bachelor]. In the 



case of Fig.16, the material collecting time is 2000 No- 
vember, with the desired amount being 0 yen (free of 
charge). 

[0356] When such sponsorship request is agreed to 
s the sponsor business operator, inclusive of its staff 
members, acts on (clicks) the button 36K. At this time 
the response result data 32 shown in Fig.11 is transmit- 
ted from the sponsorship request receiving device 22 to 
the brokerage business operator or to the sponsorship 
™ client. If the sponsorship request is not agreed to, the 
sponsor business operator acts on the button 32L. At 
this time, response result data, comprised of data stat- 
ing non-agreement to the sponsorship request, is trans- 
mitted from the sponsorship request receiving device 22 
to the brokerage business operator or to the sponsor- 
ship client. 

[0357] If the number of the display items of the request 
content confirmation form 36 is lesser than that of the 
sponsorship request condition data received, the tenor 
20 to the effect that some of the information is not displayed 
is demonstrated on the display picture so that all infor- 
mation can be confirmed through other routes. 
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(a-3) Processing operations 



[0358] When the sponsorship request condition data 
26 is received, the controller 22D displays the informa- 
tion contained in the data on the display picture based 
on the request content confirmation form 36 stored in 
30 the request content confirmation form recording unit 
22B. Meanwhile, if the request content confirmation 
form 36 is included in the sponsorship request condition 
data 26, the infoirnation is displayed using the conf irma- 
tion form. The request content confirmation form 36 may 
35 be inseparably unified to the sponsorship request con- 
dition data 26, or may be contained therein separately 
and independently. 

[0359] If the staff members of the business operator 
confirm the content of the sponsorship request on the 
<o display picture and feel that the sponsorship request 
may be agreed to, the staff members input the effect of 
agreeing to the sponsor through e.g., an icon indicated 
on the display picture. The intention to agree to the 
sponsor is returned from the controller 22D to the net- 
45 work. If otherwise, the staff members input that effect 
through e.g., an icon indicated on the display picture. 
This intention of not agreeing to the sponsor is returned 
from the controller 22D to the network. 

50 (b) Registration information inputting device 
(b-1) Terminal structure 



[0360] Referring to Fig.1 7, am embodiment of a pre- 
ss registration information inputting device 39, having the 
functions of registration necessary to receive the spon- 
sorship request distributed, is now explained. The pre- 
registration information inputting device 39 includes a 
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display controller 39A, a general item registration form 
recording unit 39B, a sponsorship condition pre-regis- 
tration form recording unit 39 C, a desired distribution 
condition recording unit 39D, an input data recording 
unit 39E, a controller 39F, a communication unit 39G, a s 
bus 39H, an operating input unit 39! and display means 
39J. 

[0361] In the case of Fig.17, the display means 39J 
and the operating input unit 39! are shown externally 
connected to the main body unit of the device. However, 
the operating input unit and the display means may be 
unified to the main body unit, as in the case of the note- 
book personal computer, a PDA or a portable telephone 
set. Additionally, the recording and communication 
units, shown enclosed in the main body unit, may be 
connected externally of the main body unit. In the case 
of the external connection, connection may be wired or 
wireless. The structures of the respective functional 
units are hereinafter explained. 

[0362] Similarly to the above-described other display 
controllers, the display controller 39A is means for re- 
producing and outputting the received or reproduced 
video data in a signal format (data format) complying 
with the display means 22H. The same applies for the 
signal format and the decoding functions of the display 
controller 39A. 

[0363] The general item registration form recording 
unit 39B is a recording or storage medium for recording 
the registration form relevant to general registration in- 
formation recorded on the membership database 21 , 
such as management information, goods-related infor- 
mation or service-related information. However, the me- 
dium may be used simultaneously for recording or stor- 
ing other information without being used exclusively as 
registration form. The medium may, for example, be a 
CD-ROM, a magnetic recording type medium, such as 
tape, floppy disc, hard disc or a magnetic card, an optical 
recording medium, such as write-once CD, DVD or op- 
tical cards, magneto-optical recording type medium, 
such as DVD, MD or MO, and a semiconductor memory, 
such as non-volatile memory or RAM. 
[0364] The registration form recorded will be ex- 
plained later in detail. The registration form may be pre- 
fixed on the medium, duplicated from other recording 
mediums or downloaded over a communication net- 
work, such as Internet. The registration form may be ex- 
panded on a video RAM provided in e.g., the display 
controller 39A. 

[0365] The sponsorship condition pre-registration 
form recording unit 39C is a recording or storage medi- 
um for recording a registration form used for pre-regis- 
tering the condition for sponsor business operators to 
agree to the sponsorship in the membership database 
21 . This medium may again be used simultaneously for 
recording or storing other information without being 
used exclusively as registration form. Specified exam- 
ples of the medium are similar to those of the general 
item registration form recording unit 39 B. 



[0366] The registration form recorded will be ex- 
plained later in detail. The registration form may be pre- 
fixed on the medium, duplicated from other recording 
mediums or downloaded over communication network, 
such as Internet. The registration form may be expand- 
ed on a video RAM provided in e.g., the display control- 
ler 39A. 

[0367] The desired distribution condition recording 
unit 39 D is a recording or storage medium for recording 
the requirements for further nan-owing the area of dis- 
tribution specified by the information on the general reg- 
istration. This medium may again be used simultane- 
ously for recording or storing other information without 
being used exclusively for recording the desired condi- 
tions for distribution. Specified examples of the medium 
are similar to those of the general item registration form 
recording unit 39B. 

[0368] Although the registration form recording unit 
for the desired conditions for distribution is not shown in 
Fig.17, it is possible to use the registration form for in- 
putting the desired conditions. In such case, it suffices 
if a registration form similar to the sponsorship request 
condition pre-registration form is used. Meanwhile, the 
desired conditions for distribution are registered by the 
sponsor business operator, inclusive of its staff mem- 
bers, using the operating input unit 39L 
[0369] The input data recording unit 39 E is a record- 
ing or storage medium for recording data input through 
the operating input unit 39I. For this medium, what has 
been said for the above-mentioned mediums applies. 
That is, this medium may again be used simultaneously 
for recording or storing other information without being 
used exclusively for recording the input data. Specified 
examples of the medium are similar to those of the gen- 
eral item registration form recording unit 39 B. 
[0370] The controller 39F is means for executing the 
control or signal processing of respective parts making 
up the pre-registration information inputting device 39 
and is generally of the micro-computer configuration. 
The functions of the controller 35A are usually pre- 
scribed by the operation system program and by the ap- 
plication program. 

[0371] The communication unit 39G is means for 
communication with outside over a network. So, com- 
munication equipment that is in keeping with the net- 
work is used. The network to be used is as described 
above. The bus 39H is a data input/output bus and may, 
for example, be an ISA bus, an EISA bus, a PCI bus etc. 
[0372] The operating input unit 39I is used for input- 
ting the items of registration or imparting various com- 
mands to the pre-registration information inputting de- 
vice 39. The operating input unit 39! may be exemplified 
by a keyboard, a mouse, a pointing device, such as pad, 
pen or pointer, a dial key, a pressing button key etc. The 
display means 39J is a device used for displaying the 
sponsorship request conditions or other information. 
Fig.1 7 shows a case where there is connected in circuit 
a display unit for a television receiver or a monitor de- 



ls 



20 



25 



30 



35 



40 



45 



50 



35 



69 



EP 1 335 592 A1 



70 



vice. 

[0373] The pre- registration information inputting de- 
vice 39, shown in Fig.17, may be formed by computer 
terminals, such as notebook or desktop personal com- 
puters, PDA, portable telephone sets, game machines 
set top boxes, PC cards, domestic electrical utensils' 
VTR, television receivers, car navigation systems, wrist 
watches having the information processing functions 
distribution terminals for music or games, financial ter- 
minals (ATM) etc., in addition to the so-called dedicated 
terminals. 

(b-2) General item registration form picture 

[0374] Fig. 18 shows an example of a general item 
registration form picture. The rudimentary information to 
be equipped in the membership database 21 may be 
written simply on writing the information in the registra- 
tion form. In the current state of network development 
m which the sponsorship request may be expected to 
be made not only in the sponsorship transaction be- 
tween different operators, but also between the operator 
and the consumer and between the individuals the ne- 
cessity for an associated interfacing technology is being 
felt keenly. y 

[0375] The registration form 40 of Fig.1 8 includes as 
its input items, a registrant name 40A, business sorts 
40B, place of residence 40C, contact site (telephone) 
40D, contact site (mail address) 40E, contact site (URL) 
40F, the GPS information 40G, place of furnishment 
40H, name of commodities or services 40I, producer 
name 40J, and Input columns 40K corresponding to re- 
spective input items. Meanwhile, the transmission but- 
ton 40L is also displayed on the picture. 
[0376] These items are recorded in the aforemen- 
tioned membership database 21 (Fig.13). The divisions 
of the commodities or services or the mail number may 
also be used as input items. Meanwhile, if a keyboard 
is used as inputting means, examples of input entries 
represented in respective input items may be selectively 
entered to respective input columns 40K associated 
with respective input items. If the transmission button 
40L is actuated (clicked), the information recorded in the 
input data recording unit 39E is sent to a terminal which 
supervises the membership database 21 . 



case of sponsorship transaction between operators and 
individuals and between individuals. 
[0378] The registration form 42 of Fig.1 9 includes as 
its input entries, names of the sponsor 42A, commodi- 
5 ties or services that can be offered for sponsorship 42B 
sponsorship condition 42C to 42E, amount requested 
42F and input entries 42G associated with the respec- 
tive input items. A transmission button 42H is also indi- 
cated on the display picture. 
10 [0379] These entries are recorded as the conditions 
under which to accept the sponsorship request 21T in 
the membership database 21 (Fig.13). Meanwhile, if the 
keyboard is used as inputting means, examples of input 
entries indicated in the respective input items may be 
w selectively input in the respective input columns 42G as- 
sociated with the respective input items. If, when the 
transmission button 42H is operated (clicked), all entries 
orthe basic information have been input, the information 
recorded in the input data recording unit 39E is sent to 
the terminal in charge of the membership database 21 
[0380] If the registration form of the same sort is used 
for registration of the desired conditions for distribution 
21 A, the registered information is recorded as the de- 
sired conditions for distribution 21 A in the membership 
25 database 21 (Fig.13). 

(b-4) Processing operations 



(b-3) Sponsorship condition registration form picture 

[0377] Fig. 1 9 shows an example of the sponsorship 
condition registration form picture. By writing the infor- 
mation in such registration form, prioritized distribution 
may be guaranteed insofar as the sponsorship request 
conditions are met. The items of the registration form 
are desirably matched to those of the sponsorship re- 
quest for use for verifying the coincidence to the input 
form of the sponsorship request. The use of such reg- 
istration enables effective registration not only in case 
of sponsorship transaction between operators but in 



[0381] When commanded to register the general in- 
30 formation for registration by an operator, the controller 
39F causes a registration form 40 held in the general 
item registration form recording unit 39B to be read out 
and sent to the display controller 39A for demonstration 
on the display picture of the display means 39J The op- 
35 erator then inputs the rudimentary information relevant 
to the sponsor business operator in accordance with the 
input Items displayed on the display picture. The input 
information is transiently stored in the input data record- 
ing unit 39E. When the input is finally determined by the 
40 operation on the transmission button 40L, the input in- 
formation is sent to the membership database 21 
through the communication unit 39G. 
[0382] The processing of pre-registration of the spon- 
sorship condition and the processing for registration of 
*s the desired conditions for distribution are similar to the 
processing of registration of the general registration in- 
formation, with the sole difference being that the regis- 
tration form displayed on the display picture is changed 
to the sponsorship condition registration form 42. 

50 

(B-4) Summary 

[0383] By constructing the sponsorship transaction 
support system, described above, it is possibleto realize 
55 effective registration not only in case of sponsorship 
transaction between operators but in case of sponsor- 
ship transaction between operators and individuals and 
between individuals. In particular, since the area of dis- 
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tribution of the sponsorship request may be freely 
changed in keeping with the intentions entertained by 
the client, brokerage business operator and the sponsor 
business operator, there may be provided a system con- 
venient to use for any of the participants in the sponsor- 
ship transaction. Moreover, the results of the sponsor, 
that is the access trends of the consumers to the spon- 
sorship transaction or the number of the concluded 
sponsorship agreements may be reflected, thus ena- 
bling the conclusion of projects of high sponsorship ef- 
fect and efficient furnishment of opportunities for public- 
ity or advertisements. 

[0384] Additionally, the information stored in the spon- 
sorship transaction may be secondarily exploited in the 
subsidiary information data creating system or in the 
digital picture program receiving and reproducing sys- 
tem, as later explained, to eliminate the deficiency of 
inputting the redundant information in the respective 
systems. In particular, it may be expected to improve the 
creation efficiency because plentiful data ancillary to the 
program is essential in realizing data broadcast attrac- 
tive to ultimate users. 

(C) Subsidiary information data creating system 

[0385] As the subsidiary information data creating 
system, there may be envisioned a system for primary 
inputting, which may be exclusively used on the site of 
creation of content material, and a system for secondary 
inputting, which may be used on the site of finishing the 
content material to the content on editing and process- 
ing. 

[0386] Meanwhile, the former is used exclusively in a 
specialized terminal work site, such as site of material 
collection, site of imaging or the site of performance, 
where there is relatively little time allowance left for data 
inputting. On the other hand, the latter is used on a site 
where there is relatively sufficient time allowance for da- 
ta inputting, such as a work site for affording secondary 
value to the created content material, such as broad- 
casting stations or video creating companies. This clas- 
sification is, however, for the sake of convenience only 
and is not to be construed in a limiting sense. 
[0387] The picture program creating system which, 
through effective utilization of the information created in 
the aforementioned sponsorship transaction support 
system, that is the sponsorship transaction related in- 
formation registered in the membership database 21 , al- 
lows for construction of a more efficient subsidiary infor- 
mation data creating system, is hereinafter explained. 

(C-1) Subsidiary information data 

(1) Basic structure 

[0388] Fig.20 shows an example of the subsidiary in- 
formation data. Specifically, Fig.20 shows the relation- 
ship between the picture program data 44 (content) and 



the subsidiary information data. The picture program da- 
ta 44 is comprised of program data (essence) 45, as 
main data, and data ancillary to the program (metadata) 
46. 

5 [0389] The program data 45 in the picture program is 
made up by video data 45A and acoustic data 45B. In 
the case of a radio program, the program data 45 is 
made up only of acoustic data. The data ancillary to the 
program 46 is made up by general key data 46A, man- 
to agement data 46B, data for goods and services 46C, 
layout information 46D and an index picture 46 E. It is 
noted that the layout information 46 D or the index pic- 
ture 46E is not essential but only optional. 
[0390] The general key data 46A is a data area where 
*5 the basic information relevant to program data itself is 
recorded. For example, UMID, meta-pocket ID (meta- 
mall ID), scene numbers, copyright, performers, posi- 
tions and sites, inclusive of the GPS information, are re- 
corded. 

[0391] The UMID is an example of the code informa- 
tion that is able to uniquely identify the image material 
on the global range, for example, on the station basis, 
on the keiretu basis and on the nation basis. Meanwhile, 
the information specifying the site may be exemplified 
by scene numbers, shot numbers, taking numbers and 
reel numbers. The meta-pocket ID gives an identifica- 
tion number of the meta-pocket corresponding to the 
scene. This information is generally accorded at the ed- 
iting stage. The meta-mall ID gives the identification 
number of the meta-mall corresponding to the scene. 
This information is also accorded at the editing stage. 
Meanwhile, the information mainly specifying the pho- 
topolymerization site is input. 

[0392] The management data 46B is a data area 
where e.g., a broadcasting station records the informa- 
tion necessary for managing video resources, such as 
date and time of real broadcast, program frame name, 
program name, contract registration date, term of valid- 
ity, conditions and so forth. Among the conditions, there 
is a condition of use as 'commercial 1 . 
[0393] The data for goods and services 46C is a data 
area in which to record the information relevant to com- 
modities or services appearing in a corresponding 
scene, such as brand name, name of commodity cate- 
gory, name of commodities or services, name of spon- 
sor business operator, contact site, URL, mail address- 
es, number of inventories, number of times of access 
events, term of delivery, price or the GPS information. 
[0394] The above information is recorded as the ne- 
cessity arises, such that not all items are always record- 
ed. Basically, a series of the information pertinent to 
each of the commodities or services is recorded. The 
data for goods and services 46C are not necessarily lim- 
ited to the subject-matter of the sponsorship or to pro- 
gram furnishment. That is, the information can be re- 
corded for ail commodities and services. It is noted that 
mainly the information for specifying the store where the 
goods can be purchased or the store where the services 
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can be furnished is input as the GPS information. 
[0395] The layout information 46D is the information 
for presenting the content of the data ancillary to the pro- 
gram 46 to the audience, and is provided especially for 
representation of meta-pocket or meta-mall. This layout s 
information 46D is sometimes as one with and insepa- 
rable from the data for goods and services 46C. Here, 
the layout information is represented as being separable 
from the data for goods and services. 
[0396] The index picture 46E is a picture for conven- 10 
fently ascertaining the content of a corresponding scene 
of the data ancillary to the program. Usually, a repre- 
sentative frame picture (still picture) of a corresponding 
scene is recorded. It is noted that the index information 
does not have to be the recorded video data but may be 15 
the link information to the video data for a corresponding 
scene. In the data area, there may be recorded the link 
information for reproducing the moving picture data or 
acoustic data. 

20 

(2) Specified example of commodity or service data 



hereinafter explained. 



(C-2) System for primary inputting 

[0401] First, the primary inputting system, which as- 
sumes the work site of creating a video material and the 
initial inputting operation of the data ancillary to the pro- 
gram, is explained. It is noted that the primary and sec- 
ondary systems are assumed only for convenience, that 
is, the primary system now explained may be used as 
the secondary system, or the secondary system may be 
used as the primary system. Specifically, no limitations 
are imposed as to the purposes of use of the respective 
systems. 

(a) Terminal structure 



[0397] Rg.21 shows specified examples of the infor- 
mation recorded as the data for goods and services. Fig. 
21 shows only the goods and services felt to appear fre- 25 
quently. Of course, these goods and services are not 
limitative. 

[0398] In the picture programs for motion pictures 
dramas or varieties (Fig. 21 A), the apparel, accessories,' 
makes, hair styles, fittings, small articles etc., worn by so 
the performer, are recorded. In a sports program (Fig. 
21 IB), logos or other commercial marks for business en- 
tities, represented on sports equipment or advertise- 
ment plate or tower, sports goods etc. are recorded. In 
addition, as the information pertinent to music and video 35 
(Fig.21C), the information pertinent to music numbers, 
such as theme songs, sound tracks or BGM, or the in- 
formation relevant to video files cited in the programs 
are recorded. 

[0399] By recording the above information, as the da- 40 
ta ancillary to the program, in association with the pro- 
gram data, expectations may be entertained of improv- 
ing the commercial value of the program data itself. That 
is, not only the picture program aimed to sell certain 
specified goods (single goods or a set of goods) or serv- 45 
ices, or so-called television, but all other sorts of picture 
programs, can be deemed as a platform that can be 
linked to the sale of the commodities or services. 
[0400] Although this may be realized by an operation 
of inputting and overseeing a large quantity of the infor- so 
mation in a short time, marked reduction in the required 
labor and economic system management become pos- 
sible by effective utilization of the information stored in 
the aforementioned sponsorship transaction support 
system. An illustrative structure of a supplementary in- 55 
formation data creating system, which takes into ac- 
count the effective utilization of the information accumu- 
lated in the sponsorship transaction support system, is 



[0402] Figs.22 and 23 mainly show an embodiment 
used for a portable program ancillary data creating de- 
vice. Fig.24 mainly shows an embodiment used for a 
standstill type program ancillary data creating device 
Meanwhile, the difference between the program ancil- 
lary data creating device shown in Figs.22 and 23 and 
that shown in Fig.24 is that the former captures the 
scene determining data directly from an imaging camera 
during the image pickup operation, while the latter cap- 
tures it from the reproduced picture. 
[0403] Fig.22 showing the basic functional structure 
of the program ancillary data creating device is first ex- 
plained. A program ancillary data creating device 50 in- 
cludes a display controller 50A, an inputting form record- 
ing unit50B, an input data recording unit20C, a gener- 
ating unit for generating data ancillary to the program 
SOD, a recording unit for recording data ancillary to the 
program 50E, a controller 50F, a code information re- 
ceiving unit 50G, a scene recording unit 50H, a selection 
input information recording unit50l, a bus50J, an oper- 
ating input unit 50K and display means 50L. 
[0404] In Fig.22, the operating input unit 50K and the 
display means 50L are shown connected to outside the 
main body unit of the device. Alternatively, the operating 
input unit and the display means may be unified with the 
main body unit of the device, as in the case of the note- 
book personal computer, PDA or a portable telephone 
set. Additionally, the recording and code communication 
units, shown enclosed in the main body unit, may be 
connected externally of the main body unit. For external 
connection, cable connection or radio connection may 
be used. For cable connection, serial connection or par- 
allel connection may be used, while coaxial cables, 
twisted pair cables or optical fibers may be used. For 
radio connection, there is no particular limitation to the 
radio frequency range, while the spread spectrum sys- 
tem, such as Bluetooth, may or may not be used. 
[0405] The display controller 50A is means for repro- 
ducing and outputting video data in signal forms (data 
forms) conforming to the display means 50L. Output sig- 
nals may, for example, be a video output (NTSC video, 
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composite video and S-video) or RGB output. The dis- 
play controller 50A includes a data decoding function, 
such as one provided by MPEG (Moving Picture Experts 
Group), depending on the purposes of use. 
[0406] The inputting form recording unit SOB is a re- 5 
cording or storage medium for storing an input form for 
data ancillary to the program. It is noted that the medium 
does not have to be used solely for storage of the input 
form of the data ancillary to the program, but may be 
used for recording or storing other information in com- 
bination. The medium may, for example, be a CD-ROM, 
a magnetic recording type medium, such as tape, floppy 
disc, hard disc or a magnetic card, an optical recording 
medium, such as write-once CD, DVD or optical card, 
magneto-optical recording type medium, such as DVD, 
MD or MO, and semiconductor memory, such as non- 
volatile memory or RAM. 

[0407] The inputting form recorded will be explained 
later in detail. The registration form may be pre-fixed on 
the medium, duplicated from other recording mediums 
or downloaded over a communication network, such as 
Internet. The registration form may be expanded on a 
video RAM provided in e.g., the display controller 50A. 
[0408] The input data recording unit 50C is a record- 
ing or storage medium for recording data input via op- 
erating input unit 50K. By and large, a RAM is used as 
this medium. However, there is no limitation to the sort 
of the medium provided that the medium used is one on 
which it is possibieto write inputdata. On the other hand, 
it is not mandatory for this input data recording unit 50C 
to be a recording unit dedicated to recording the input 
data. For example, the input data recording unit 50C 
may be realized by a medium which is the same medium 
as the inputting form recording unit50B, with the record- 
ing area then being a distinct area. The inputting form 
recording unit 50 B or the input data recording unit 50C 
may be provided within the inside of the display control- 
ler 50A. 

[0409] The generating unit 50 D for generating data 
ancillary to the program is means for linking the input 
data recorded in the input data recording unit 50C and 
the code information recorded in the scene recording 
unit 50H, such as UMID or time codes as the scene in- 
formation recorded in the scene recording unit 50H, to 
generate data ancillary to the program. It is however 
possible to cause the program ancillary data generating 
unit 50D to generate data ancillary to the program 46 
shown in Fig.20. It may however be presumed that the 
generating unit 50D for generating data ancillary to the 
program for a site of imaging placed under temporal 
constraint or a similar creating site has as an object of 
creation to create the program ancillary data consisting 
only of the limited information such as the information 
specifying the scene or the company furnishing com- 
modities or services. 

[0410] The recording unit 50 E for recording data an- 
cillary to the program is a recording or storage medium 
for recording the created program ancillary data. The 



medium may be exemplified by a CD-ROM, a magnetic 
recording type medium, such as tape, floppy disc, hard 
disc or a magnetic card, an optical recording medium, 
such as write-once CD, DVD or an optical card, a mag- 
neto-optical recording type medium, such as DVD, MD 
or MO, and a semiconductor memory, such as non-vol- 
atile memory or RAM. 

[041 1] If the program ancillary data recorded is to be 
transmitted to other devices, such as, for example, a de- 
vice for processing or editing the data ancillary to the 
program, over a communication system, which may be 
wired or wireless, it is sufficient if communication means 
having the communication function for the program an- 
cillary data is provided in the program ancillary data gen- 
erating unit. It is noted that the communication function 
complying with the transmission medium is required of 
this communication means. When a medium separable 
from the main body unit of the device is used for a re- 
cording medium or a storage medium, such as DVD or 
non-volatile memory, the medium can be taken out from 
the main body unit of the device and transferred to an- 
other device. 

[0412] The controller 50 F is means for executing var- 
iegated control or signal processing of various compo- 
nent parts making up the generating device 50 for data 
ancillary to the program, and is routinely of the micro- 
computer configuration. The functions of the controller 
50F is usually prescribed by an operation system pro- 
gram and by an application program. 
[0413] The code information receiving unit 50G is 
means for receiving the scene specifying information, 
such as UMID or time codes, of program data notified 
in real-time from an imaging camera 51 in the course of 
the imaging operation. The functions required of the 
code information receiving unit 50G are those conform- 
ing to the transmission system between the imaging 
camera 51 and the main body unit of the device. Al- 
though the communication system shown in Fig.22 is a 
wireless system, it is of course not contemplated to ex- 
clude the wired communication system. It should be not- 
ed that the wireless system is more convenient since 
there is imposed no constraint as to inputting positions, 
and also since there is no necessity of providing a wired 
cable. 

[041 4] It is also possible to provide means for detect- 
ing a scene change point (a scene change extraction 
unit) in the code information receiving unit so that the 
new code information will be automatically recorded in 
the scene recording unit50H every time a scene change 
point is detected. Meanwhile, if the scene specifying in- 
formation is UMID, one code is associated with one 
scene, so that the scene change point may be detected 
by monitoring the scene change point of the code infor- 
mation. 

[0415] The scene recording unit50H is a recording or 
storage medium for writing the code information, being 
captured by the code information receiving unit 50G, 
based on the scene information capturing signal accord- 
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ed by the operating input unit 50K at the time of inputting 
the signal. The medium may be exemplified by a 
CD-ROM, a magnetic recording type medium, such as 
tape, floppy disc, hard disc or a magnetic card, an optical 
recording medium, such as write-once CD, DVD or op- 
tical card, magneto-optical recording type medium, such 
as DVD, MD or MO, and semiconductor memory, such 
as non-volatile memory or RAM. It is of course possible 
to use the physically same medium as the other record- 
ing unit, with only the recording area being then different 
from that used by the other recording unit. 
[0416] The selection input information recording unit 
50! represents the function provided when a selective 
input function is readied for inputting the data ancillary 
to the program. So, this recording unit is unnecessary 
when all the inputting operations are executed manually. 
The information recorded in the recording unit may be 
the information recorded in advance by an operator of 
the generating device 50 for data ancillary to the pro- 
gram, or the information pertinent to the commodities or 
services procured from the aforementioned sponsor- 
ship transaction support system or the database sys- 
tem. As methods for procuring the information from the 
sponsorship transaction support system or the data- 
base system, a method of having the data furnished by 
communication and a method of having the data fur- 
nished by a medium can be used. If the communication 
is used, it may be wired or wireless. 
[0417] The information at issue may be the totality of 
the information registered in connection with the com- 
modities or services. When the number of the items of 
selection is excessive, the operating efficiency may oc- 
casionally be lowered. Consequently, when the use on 
the site is presupposed, it is possibly desirable to limit 
the information to the name of the company furnishing 
the commodities or services or to the common names 
of the commodities or services furnished. It is noted that, 
except in case the information is input in parallel with 
the imaging operations, selection may be made up to 
more detailed information, by making contrivances in re- 
trieving characteristics of the selection items, so that, in 
such case, such information may be recorded which is 
in meeting with the purposes of use in question. 
[0418] The medium may be exemplified by a 
CD-ROM, a magnetic recording type medium, such as 
tape, floppy disc, hard disc or a magnetic card, an optical 
recording medium, such as write-once CD, DVD or op- 
tical card, a magneto-optical recording type medium, 
such as DVD, MD or MO, and a semiconductor memory,' 
such as non-volatile memory or RAM. It is of course pos- 
sible to use the physically same medium as the other 
recording unit with only the recording area being then 
different from that used by the other recording unit. If the 
information is furnished in the form of a medium, it may 
be contemplated to use a CD-ROM or a non-volatile 
memory. 

[041 9] The bus 50J is a data input/output bus, which 
may, for example, be an ISA bus, an EISA bus or a PCI 



bus. The operating input unit 50K may, for example, be 
a keyboard, a mouse, a pointing device, such as pad, 
pen or pointer, a dial, having or not having a click func- 
tion, and a pushbutton key. 
5 [0420] The display means 50L is a device used for 
demonstrating a picture, such as an input screens for 
data ancillary to the program. In the case of a portable 
device, a small-sized lightweight display device, such 
as a liquid crystal device, may be used. When the dis- 
10 play means 50L is used as a standstill type device, a 
monitor device or a television receiver may, of course 
be used. 

[0421] The imaging camera 51 may be made up by a 
camera by itself, or a camera fitted with a recorder (so- 
15 called camera built-in type recorder), it may also be an 
electronic camera for photographing a still picture. 
Meanwhile, the generic key data pertinent to imaging is 
recorded on a preset recording medium along with pro- 
gram data. 

20 [0422] Reference is now made to Fig.23 showing a 
further input assisting function to the program ancillary 
data generating device in Fig.23. Since the parts corre- 
sponding to those shown in Fig.22 are indicated with the 
same reference numerals as in Fig.22, only different 
25 functional portions are explained. The added portions 
are four units, that is a sponsor information decision unit 
50M, a recording unit for sponsorship commodities or 
services 50N, a flag imparting unit 50O and a check im- 
parting unit 50P. 
30 [0423] The sponsor information decision unit 50M is 
means for verifying whether or not there is any portion 
in the information recorded in the recording unit for 
sponsorship commodities or services 50N which coin- 
cides with the input information on the commodities or 
35 services. This function is particularly effective when the 
information on specified commodities or services is in- 
put. Meanwhile, the function of the decision unit is rou- 
tinely implemented as one of application programs ex- 
ecuted by the controller 50F. 
40 [0424] The recording unit for sponsorship commodi- 
ties or services 50N is a recording or storage medium 
which includes the recording information for decision by 
the sponsor information decision unitSOM. This medium 
may be exemplified by CD-ROM, a magnetic recording 
« type medium, such as tape, floppy disc, hard disc or a 
magnetic card, an optical recording medium, such as 
write-once CD, DVD or an optical card, a magneto-op- 
tical recording type medium, such as DVD, MD or MO, 
and a semiconductor memory, such as non-volatile 
50 memory or RAM. It is of course possible to use the phys- 
ically same medium as the other recording unit with only 
the recording area being then different from that used 
by the other recording unit. Consequently, the same me- 
dium may be used for this recording unit for sponsorship 
55 commodities or services and for e.g., the selection input 
information recording unit 50I. Since this information is 
the information on the sponsorship- related commodities 
or services, agreed to as to the creation of a picture pro- 
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gram, the relevant information accumulated in the 
above-mentioned sponsorship transaction support sys- 
tem, if any, may be used for realizing effective informa- 
tion utilization. However, when the sponsorship trans- 
action support system is not used, the information sep- 
arately input may be used. 

[0425] The flag imparting unit 50O is means for im- 
parting to the program ancillary data flag data indicating 
that the input information has been found by the sponsor 
information decision unit 50M to be the sponsorship-re- 
lated commodities or services. Specifically, the flag im- 
parting unit is means for setting the data portion, pro- 
cured for displaying whether or not the commodities or 
services in question are those pertinent to the accepted 
sponsorship, to a significant level (imparting a flag). By 
recording the information in question in the data ancil- 
lary to the program, signal processing of preparing a pic- 
ture consisting only of a collection of the sponsorship- 
related information becomes feasible. 
[0426] The check imparting unit 50P is means for 
causing the same information as the flag imparting unit 
50O to be reflected in the display picture. That is, the 
check imparting unit 50P is means for automatically im- 
parting check marks to check item columns on the dis- 
play picture if, as a result of the decision by the sponsor 
information decision unit 50M, the input information cor- 
responds to the sponsorship-related commodities or 
services. It is of course presupposed that there is a dis- 
play column in the input item that is used for indicating 
whether or not the information in question is pertinent to 
the sponsorship. It is however also possible that the in- 
formation is manually input by no other than the opera- 
tor. This function, if provided, is helpful to improve the 
inputting efficiency and to evade mistaken inputs. 
[0427] Meanwhile, the flag imparting unit 50O and the 
check imparting unit 50P are each implemented as one 
of the application programs executed by the controller 
50 F. These functions are loaded as one set in conjunc- 
tion with the sponsor information decision unit 50M. 
While these functions are usually loaded on both units, 
it may also be loaded on one of the units. 
[0428] Reference is made to Fig .24 showing an illus- 
trative structure of a generating device for data ancillary 
to the program, adapted for acquiring the scene infor- 
mation, for which the data ancillary to the program is to 
be prepared, from the general key data reproduced from 
the recording medium. Similarly to Fig .22, Fig.24 shows 
only the basic functional portions. It is therefore possible 
to load the sponsor information decision unit 50M, a re- 
cording unit for sponsorship commodities or services 
50N, a flag imparting unit 50O and a check imparting 
unit 50P shown in Fig.23. 

[0429] The points of difference of the program ancil- 
lary data generating device of Fig.24 from the device 
shown in Fig. 22 reside in the substitution of a code in- 
formation extraction unit 50Q for the code information 
receiving unit 50G, and in a picture program data repro- 
ducing device 50R being shown in the drawing. 



[0430] The code information extraction unit 50Q is 
means for perpetually monitoring replay signals of the 
picture program data reproducing device 50R for ex- 
tracting the scene specifying information, such as UMID 

5 or time codes. Meanwhile, in the case of the generating 
device for generating the data ancillary to the program, 
shown in Fig.24, a replay picture needs to be displayed 
on the picture of the display means 50 or on the picture 
of other monitor devices. An operator who intends to 

w prepare data ancillary to the program executes the op- 
eration of scene selection at such a scene as is suited 
to the creation of the program ancillary data pertinent to 
the commodities or services as the operator confirms 
the picture being reproduced on a display picture in 

15 question occasionally with speech or other sound. It is 
possible to provide a scene change extraction unit in the 
code information extraction unit 50Q so that the new 
code information can be automatically recorded in the 
scene recording unit 50H each time a scene change 

20 point is detected. 

[0431] The picture program data reproducing device 
50R is a device for reproducing picture program data 
recorded on a medium exemplified by a CD-ROM, a 
magnetic recording type medium, such as tape, floppy 

25 disc, hard disc or a magnetic card, an optical recording 
medium, such as write-once CD, DVD or optical card, 
magneto-optical recording type medium, such as DVD, 
MD or MO, and a semiconductor memory, such as non- 
volatile memory or RAM, and includes a variable repro- 

30 ducing signal system loaded thereon, depending on the 
types of the mediums or on the recording formats. 
[0432] It is noted that the picture program data repro- 
ducing device 50R need not be a replay-only device but 
may have loaded thereon a recording system adapted 

35 for recording picture program data supplied from the im- 
aging camera 51 on a preset medium in accordance with 
a preset format. Although the picture program data re- 
producing device 50R is shown in Fig.24 to be provided 
within the casing of the generating device 50 for pro- 

40 gram ancillary data, it may, of course, be mounted out- 
side of the main body unit of the device. In this case, the 
main body unit of the device needs to be provided with 
a function of communication with the picture program 
data reproducing device 50 R. Meanwhile, if the picture 

45 program data reproducing device 50 R is provided out- 
side of the main body unit of the device, the generating 
device for program ancillary data 50 may readily be de- 
signed as a portable device. 

[0433] The generating device 50 for program ancillary 
50 data, shown in Figs. 22 to 24, may be formed by compu- 
ter terminals, such as notebook or desktop personal 
computers, PDA, portable telephone sets, game ma- 
chines, set top boxes, PC cards, domestic electrical 
utensils, VTR, television receivers, car navigation sys- 
55 terns, wrist watches having the information processing 
functions, distribution terminals for music or games, fi- 
nancial terminals (ATM) etc., in addition to the so-called 
dedicated terminals. 
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(b) Input form picture 



[0434] Figs.25 to 27 show examples of input form pic- 
tures. These input forms are suited to a work site where 
especially the inputting time available is limited. By em- 
ploying these input forms, input items can be unified no 
matter who is performing the inputting operations. In 
particular, since a large number of staff members and 
program creating companies are involved in the creation 
of a picture program, there is a significant merit of uni- 
fication of input items in overseeing picture resources 
By positively exploiting these properties, it is possible to 
distribute the inputting operations among plural opera- 
tors to realize efficient inputting. 
[0435] Fig.25 shows a first instance of the input pic- 
ture. In the case of Fig.25, an information inputting col- 
umn 54 and an already input information display column 
55 are displayed in an upper row and in a mid row of a 
display picture 53, respectively, while a decision button 
56, a correction button 57, a scene inputting button 58 
and a sponsor button 59 are displayed in a lower row 
Since a manual operation of inputting the scene is here 
presupposed, the scene inputting button 58 is indicated 
Additionally since here a manual operation of inputting 
the tenor that the goods or services are those pertinent 
to the sponsorship as furnished by the sponsor business 
operator is presupposed, there is also provided the 
sponsor button 59. 

[0436] In the information inputting column 54 there 
are displayed input items 54A to 54C and associated 
input columns 54D to 54F. In the case of Fig.25, the input 
items are a scene information 54A, a time code 54B and 
a company name 54C. Meanwhile, the company name 
is that of a company which manufactured or supplied 
the commodities or services appearing in the pictures 
of respective scenes. 

[0437] In the input columns 54D and 54E, among the 
input columns 54D to 54F, which are associated with the 
scene information 54A and the time code 54B, respec- 
tively, the code information which is valid at the time 
point of operation of the scene inputting button 58 is au- 
tomatically displayed. It is noted that, in automatically 
inputting the scene information or the time code, the 
code information is displayed automatically. Conse- 
quently, the item in need of manual inputting by an op- 
erator in the present exemplary picture is solely the com- 
pany name 54C. Meanwhile, the information as to 
whether or not the information is the information con- 
cerning the commodities or services pertinent to the 
sponsor is input by acting on the sponsor button 59 Al- 
though the setting of Fig.25 is made so that the informa- 
tion indicating whether or not the input object is the com- 
modities or services pertinent to the sponsor is not dis- 
played in the information inputting column 54, this infor- 
mation may, if desired, be displayed. 
[0438] As means for manually inputting the informa- 
tion in the information inputting column 54, an input key 
not shown, with a letter allocated thereto, a dial key or 



a keyboard etc., is used. A pen inputting function may 
also be used. For finally determining the contents en- 
tered into the information inputting column 54, the deci- 
sion button 56 is used. The information, input finally, is 
5 displayed in the already input information display col- 
umn 55. 

[0439] In the already input information display column 
55, a scroll bar 55A and a title bar 55B are demonstrated 
in addition to the already input information. In the title 
10 bar 55B, items associated with input items of the infor- 
mation inputting column 54 are normally displayed. In 
the case of Fig.25, an item not indicated in the informa- 
tion inputting column 54, that is the 'sponsorship-related 
goods', is displayed in the title bar55B, for the sake of 
confirmation of the information input by operation of the 
sponsor button 59. Alternatively, the information as to 
whether or not the goods (or services) are the sponsor- 
ship-related commodities or services can be adapted to 
be entered on the display picture of the already input 
20 information display column 55. For inputting the infor- 
mation, such methods may be contemplated which con- 
sist in operating the sponsor button59 as the check col- 
umn is designated, or which consist in touching the 
check column with an input pen, for example, a method 
25 consisting in detecting the position of a pen point by po- 
sition detection means comprised of a transparent elec- 
trode which covers up the display picture. The items 
checked are changed in display so that the items so 
checked can be discriminated from other unchecked 
30 items. In Fig.25, the check column is displayed as if it is 
painted as indicated at 55C. 

[0440] The correction button 57 is used for correcting 
the input information. For example, the correction button 
57 is operated as an input letter string desired to be cor- 
35 rected is designated. By acting on the correction button 
57, the information to be corrected is again demonstrat- 
ed in the information inputting column 54 in readiness 
for correction. 

[0441] If desired to register plural company names for 
40 one scene, the decision button 56 may be depressed 
as a button having an allocated function of inputting plu- 
ral company names, in order to save the scene informa- 
tion 54D and a time code 54E. Alternatively, there may 
be provided duplication buttons for the scene specifying 
<* information (scene information and time code in the 
case of Fig.25) for duplicating the scene specifying in- 
formation of the input information selected on the dis- 
play picture of the already input information display col- 
umn 55 in the input columns 54D and 54E. If there is 
so provided a function of automatically updating the scene 
specifying information of the input column, it is possible 
not to update the information of the input columns 54D, 
54E until such time when an operator performs a spec- 
ified operation. 

55 [0442] Fig.26 shows a second exemplary input pic- 
ture. The exemplary display, shown in Fig.26, shows a 
case wherein the selection input information recording 
unit 50I is loaded in the device 50 for generating pro- 
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gram ancillary data. In Fig.26, the function of selective 
inputting is allocated to the input column 54F for the in- 
put item 'company name'. In Fig.26, there is shown the 
state in which a selective candidate picture 54G for the 
input column 54F is shown deployed. I n the case of such s 
input picture, it is sufficient if the inputting operator se- 
lects one of the items shown in a list in a selection can- 
didate picture 54G. 

[0443] In Fig. 27, show in a third exemplary input pic- 
ture, a drag-and-drop system is used for inputting the 
information, in distinction from the aforementioned two 
exemplary input pictures. So, in this third exemplary pic- 
ture, there is adopted a system of displaying input can- 
didates in a list in a selection item display column 60. In 
the case of Fig.27, a candidate button 60C, having the 
input candidate information allocated thereto, is dis- 
played. The present exemplary input picture is the same 
as the aforementioned exemplary input picture in that a 
scroll bar 61 A and a title bar 61 B are displayed in the 
information inputting column 60. The present exemplary 
input picture is also the same as the aforementioned ex- 
emplary input picture in that the scroll bar 60A and the 
title bar 60B are demonstrated in the information input- 
ting item 60. 

[0444] Moreover, in this third exemplary input picture, 
the already input information display column 61, oper- 
ating not only as a column for confirming the already 
input information but also as a column for inputting the 
novel information, is indicated below the information in- 
putting column 60. It is noted that the scroll bar 61 A and 
the title bar 61 B are displayed in the already input infor- 
mation display column 61 as in the above-described ex- 
emplary input picture. The display of items of the check 
column of the goods pertinent to sponsor (or services 
pertinent to sponsor) are changed as indicated at 61 C, 
as in the above-described exemplary input picture. 
[0445] In the case of Fig.27, the inputting of the com- 
pany names of the commodities or services is by drag- 
and drop of the selected candidate button 60C at a dis- 
play position of the scene specifying information (scene 
information and time code) as captured by the operation 
on the scene inputting button 58 or by the automatic 
scene change detection. Instead of by drag-and drop, 
the information may be input by acting on the decision 
button 56 as e.g., a certain one of buttons of the candi- 
date buttons 60C remains selected. 
[0446] In the aforementioned three exemplary input 
pictures, it is presupposed that the site information, such 
as UMID, or the time code, such as hour: minute: sec- 
ond: frame number, are automatically input. Alternative- 
ly, these information may also be input manually. Stilt 
alternatively, one of the scene information and the time 
code may be input. The same may be said of display. 

(c) Processing operations at the time of inputting 
operations 

[0447] Figs.28 ands 29 show an exemplary process- 



ing sequence at the time of inputting by the generating 
device 50 for program ancillary data. Fig. 28 is relevant 
to the generating device 50 forprogram ancillary data 
having the selective inputting function, while Fig.29 is 
relevant to the generating device 50 for program ancil- 
lary data having the drag-and-drip function. It should be 
noted that, in both Figs.28 and 29, there is loaded the 
function of automatically affording check marks in the 
check item columns of the display picture in case the 
input information corresponds to the sponsorship-relat- 
ed commodities or services. 

[0448] Referring first to Fig. 28, when commanded to 
execute the application program for executing the func- 
tion in question, the controller 50F reads out an inputting 
form from the inputting form recording unit SOB (step 
S21). The controller 50F deploys the input form, thus 
read out, on the display controller 50A, to output the in- 
put form in a preset signal form (data format) to the dis- 
play means 50L. This demonstrates the input form 
shown for example in Fig.26 (step S22). 
[0449] At this display picture, the controller 50F veri- 
fies whether or not the operation of selecting an input 
scene has been carried out (step S23). This selective 
operation is carried out through the scene inputting but- 
ton 58. If the selection of the scene through the opera- 
tion of the scene inputting button 58 has been con- 
firmed, that is if the result at step S23 is affirmative, the 
controller 50 F records the code Information, captured at 
the time point in question, in the scene recording unit 
50H. For example, the code information capturing path 
may be through the code information receiving unit50G 
(Figs. 22 and 23) orthrough the code information extrac- 
tion unit 50Q (Fig.24). The controller 50F then imparts 
the captured information to the display controller 50A to 
display the information in relation to the input form being 
displayed. 

[0450] The controller 50F then verifies whether or not 
the input item as specified is an item that can be selec- 
tively input. A negative result and an affirmative result 
are obtained when an item directly input by an operator 
(an item where an optional letter string is input), such as 
'company name' of Fig. 25, is specified, and when an 
item directly entered by the operator, such as [compa- 
ny's name] of Fig.26, is specified, respectively. 
[0451] In case the affirmative result is obtained, the 
controller 50 F reads out the information previously pro- 
vided as to a relevant item, that is the information re- 
corded in the selection input information recording unit 
50! , to impart the information so read out to the display 
controller 50A to make a list display of input candidates, 
as shown in Fig.26 (step S26). If a candidate is selected 
in this state, that is when an item is selected by a cursor 
shifting key, not shown, and the decision button 56 is 
depressed, the controller 50F displays the information 
so decided on in the input column 54F (step S27). If the 
negative result is obtained, the controller 50 F causes 
the information input by the operator as to the relevant 
item to be displayed in the input column 54F. 
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[0452] When the input item is finally determined in this 
manner, the controller 50F causes the sponsor informa- 
tion decision unit 50M to check whether or not the input 
information is relevant to the commodities or services 
pertinent to the sponsor (step S29) . If the input informa- 
tion is verified to be the commodities or services perti- 
nent to the sponsor, that is if the affirmative result is ob- 
tained at step S29, the check imparting unit SOP issues 
a command to the display controller 50A to impart a 
check mark. Thus, the check mark, such as 55C, is dis- 
played in the check item column to complete the input 
processing for one commodity or service (step S30). 
[0453] When a decision is given that the input infor- 
mation is not the sponsorship-related commodities or 
services, that is if the negative result is obtained at step 
S29, no command is issued from the check imparting 
unit 50P to the display controller 50A, or a command is 
issued from the check imparting unit to the display con- 
troller that no check mark is to be displayed. As a result, 
the inputting processing for one commodity or service 
is completed, there being no change made in the display 
picture. 

[0454] For finally determining the termination of the 
input processing, an operation by the decision button 56 
may be made a necessary step. In Fig.28, it is assumed 
that the generating device for data ancillary to the pro- 
gram has the function of automatically verifying the com- 
modities or services relevant to the sponsorship and the 
function of automatically imparting the check mark. 
However, since the automatic check as to whether or 
not the commodities or services in question are those 
relevant to the sponsorship has no significant meaning 
if the input is on the company level, it may be contem- 
plated to make the check of the check columns of the 
items of the sponsor goods by manual operations. 
[0455] Referring to Fig.29, when commanded to exe- 
cute the application program for function execution, the 
inputting form recording unit 50B of the controller 50 F 
deploys the read-out input form on the display controller 
50A to output the input form in a preset signal or data 
format to the display means 50L. So, the input form 
shown for example in Fig.27 is displayed (step S32). 
[0456] In this display picture, the controller 50F 
checks to see whether or not the input picture selecting 
operation has been carried out (step S33). This select- 
ing operation is carried out through the scene inputting 
button 58. When the scene selection through the oper- 
ation of the scene inputting button 58 is recognized, that 
is if the affirmative result is obtained at step S33, the 
controller 50F causes the code information, captured at 
the time point in question, to be recorded in the scene 
recording unit 50H. The code information capturing 
route is the same as that explained in Fig.28. The con- 
troller 50F then sends the captured information to the 
display controller 50A to display the information in as- 
sociation with the input form being demonstrated. 
[0457] The controller 50F monitors which of the items 
demonstrated on the selection item display column 60 



has been selected by the operator and which area is the 
destination of drag-and-drop of the selected informa- 
tion. If the destination of drag-and-drop is a preset po- 
sition in the already input information display column 61 , 

5 the controller causes the selected information to be 
demonstrated at such position. The drop destination 
may be limited to a display string in which the scene 
specifying information of the information currently being 
input is displayed, or may simply be any arbitrary posi- 

10 tion in the already input information display column, 
[0458] The ensuing processing is the same as that of 
Fig.28, as may be seen from the same reference nu- 
merals used as those in Fig.28, and hence is not ex- 
plained specifically. 

15 

(C-3) Secondary inputting system 

[0459] The secondary inputting system, which pre- 
supposes the post-inputting operation of the data ancil- 

*> lary to the program or the editing worksite of the picture 
material, is hereinafter explained. As aforesaid, the di- 
vision into the primary and the secondary is for conven- 
ience only such that the secondary inputting system 
may also be used as the primary inputting system. 

25 Meanwhile, it is not mandatory that the system is pro- 
vided on the picture creating business operator, such 
that it may be provided on the brokerage business op- 
erator acting as an intermediary in the commercial trans- 
action between the audience and the sponsor business 

50 operator. 

(a) Terminal structure 



[0460] Figs.30 and 31 show an embodiment of the 
35 program ancillary data creating device and Fig.32 
shows an embodiment of a picture program editing de- 
vice provided with the program ancillary data creating 
device. The ultimate outputting of the program ancillary 
data creating device is classified into the transmission 
40 path output type and the recording output type. These 
respective cases are hereinafter explained. 
[0461] Reference is first made to Figs.30 and 31. A 
creating device for program ancillary data 63, shown in 
Fig.30, includes a display controller 63A, an input data 
« recording unit 63B, a recording unit for program ancillary 
data 63C, a controller 63D, a data creating unit for pro- 
gram ancillary data 63E, an index picture recording unit 
63F, an index picture creating unit 63G, a layout infor- 
mation holding unit 63H, a link information embedding 
so functional unit 63I, an interface 63J, a bus 63K, an op- 
erating input unit 63L and display means 63M. 
[0462] Although the operating input unit 63L and the 
display means 63M are shown in Fig.30 to be connected 
externally to the main body unit of the device, the oper- 
as ating inputting unit and the display means may also be 
connected as one to the main body unit of the device, 
as in the case of a notebook personal computer, a PDA 
and a portable telephone set. Although the recording 
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unit and the various functional units are shown enclosed 
in the main body unit, these may also be connected to 
outside the main body unit. For external connection, ca- 
ble connection or radio connection may be used, in the 
case of the cable connection, serial or parallel connec- 
tion may be used, while coaxial cables, twisted pair ca- 
bles or optical fibers may also be used. As for radio con- 
nection, there is no particular limitation to the radio fre- 
quency range used, while the spread spectrum system, 
such as Bluetooth, may or may not be used. 
[0463] The display controller 63 A is means for repro- 
ducing and outputting picture data in signals forms (data 
forms) conforming to the display means 63M. Output 
signals may, for example, be a video output (NTSC vid- 
eo, composite video and S-video) or an RGB output. 
The display controller includes a data decoding function, 
such as one provided by MPEG (Moving Picture Experts 
Group), depending on the purposes of use. 
[0464] The input data recording unit 63 B is a record- 
ing or storage medium for recording data input through 
the operating input unit 63L. While the medium routinely 
employed is a RAM, there is no limitation to the type of 
the recording medium provided that it permits writing of 
input data. There is no necessity for the input data re- 
cording unit 63B to be a recording unit dedicated to the 
recording of the input data. It is of course possible to 
use the physically same medium for the input data re- 
cording unit 63B as the recording unit for program an- 
cillary data 63C, if only the recording area is different 
from that used by the other recording unit. Meanwhile, 
the input data recording unit 63 B and the recording unit 
for program ancillary data 63C may be provided within 
the display controller 63A. 

[0465] The picture program data recording device 
63C is a device for recording program ancillary data cre- 
ated and may be exemplified by a magnetic recording 
type medium, such as tape, floppy disc, hard disc or a 
magnetic card, an optical recording medium, such as 
write-once CD, DVD or an optical card, a magneto-op- 
tical recording type medium, such as DVD, MD or MO, 
and a semiconductor memory, such as non-volatile 
memory or RAM 

[0466] The controller 63D is means for executing var- 
iegated control or signal processing of various compo- 
nent parts making up the generating device 63 for pro- 
gram ancillary data. This controller is routinely of the mi- 
cro-computer configuration. The functions of the con- 
troller 63D are usually prescribed by an operation sys- 
tem program and by an application program. 
[0467] The data creating unit for program ancillary da- 
ta 63 E is means for linking input data recorded in the 
input data recording unit 50C and the layout information 
recorded in the layout information holding unit 63H (lay- 
out information used in presentation for the audience) 
and other information to generate program ancillary da- 
ta. In this data creating unit 63E for program ancillary 
data, the program ancillary data 46, shown in Fig.20 t is 
produced. 



[0468] The index data recording unit 63F is a record- 
ing or storage medium for recording an index picture 
used in confirming a picture program scene associated 
with the program ancillary data. While the medium rou- 
5 tinely employed is a RAM, there is no limitation to the 
type of the recording medium provided that it permits 
writing of input data. There is no necessity for the index 
data recording unit 63F to be a recording unit dedicated 
to the recording of the index data. It is of course possible 
to use the physically same medium for the index data 
recording unit 63F as the program ancillary data record- 
ing unit 63C, with only the recording area being then dif- 
ferent from that used by the program ancillary data. 
[0469] The index picture creating unit 63G is means 
for extracting a typical frame picture of a scene corre- 
sponding to the program ancillary data as an index pic- 
ture. Since the index picture is used for confirming the 
corresponding scene, the picture information reduced 
in picture quality as compared to the picture used in the 
actual broadcast is produced. It is, however, not man- 
datory to degrade the picture quality, such that the same 
information may be included in the index picture as that 
of the actual broadcast. 

[0470] For the index picture, such a frame picture rep- 
resentative of the content of each scene is used. For 
example, a leading frame of a corresponding scene, a 
frame which allows for facilitated confirmation of the 
commodities or services pertinent to the sponsorship, 
as selected by the operator, or a trailing end frame of 
the corresponding scene, may be used as the index 
frame. If the rule for selecting the frame picture is set in 
this manner, it is possible for the index picture creating 
unit 63G to create the index picture automatically. How- 
ever, depending on the selection by the operator, a 
frame picture of the time code as commanded by the 
operating input unit 63K may be used. The index picture 
may also be represented by computer graphics. 
[0471 ] The layout information holding unit 63H is a re- 
cording or storage medium for holding the layout infor- 
mation used for prescribing the modifying information or 
the display position for the program ancillary data for 
facilitating the confirmation by the audience of the pro- 
gram ancillary data. These mediums may be exempli- 
fied by a magnetic recording type medium, such as tape, 
floppy disc, hard disc or a magnetic card, an optical re- 
cording medium, such as write-once CD, DVD or an op- 
tical card, a magneto-optical recording type medium, 
such as DVD, MD or MO, and a semiconductor memory, 
such as non-volatile memory or RAM. It is of course pos- 
sible to use the physically same medium as the other 
recording unit, with only the recording area being then 
different from that used by the other recording unit. 
[0472] The layout information may be exemplified by 
e.g., a style sheet of an XML (Extensible Markup lan- 
guage) file. In universality is taken into account, such 
layout information that can be handled independently of 
e.g., the detailed information pertinent to commodities 
or services is desirable. Of course, the demonstration 



is 



20 



25 



30 



35 



40 



45 



50 



45 



89 



EP 1 335 592 A1 



90 



of the program ancillary data by the HTML (Hypertext 
Markup Language) is not excluded. Which language Is 
to be used for Indicating the program ancillary data is 
determined on the basis of a route through which the 
information is furnished. 

[0473] As for the layout information, which will be ex- 
plained in detail subsequently, it may be contemplated 
to provide solely the information pertinent to one com- 
modity or service with a sole information furnishing pic- 
ture, to enable the presentation of the information per- 
tinent to plural commodities or services appearing in a 
sole scene by the sole information furnishing picture (as 
in the case of the aforementioned meta-pocket), or to 
enable the presentation of the information pertinent to 
plural commodities or services appearing in plural 
scenes, each of which is associated with the information 
pertinent to the plural commodities or services, by the 
sole information furnishing picture (as in the case of the 
aforementioned meta-mall). 

[0474] The link information embedding functional unit 
63I is means for embedding the link information used 
for linking the moving picture reproducing function of 
picture data or the reproducing function of acoustic data 
of a scene associated with the index picture in case the 
index picture is embedded in the program ancillary data. 
For example, there is recorded the link information to 
the file information of program data (picture data or 
acoustic data) having the scene specifying information 
of the corresponding scene, such as UMID or time code, 
or the link information to the file information separately 
provided for the reproducing function of the moving pic- 
ture or that of the acoustic data. Meanwhile, the data 
separately provided for the functions of reproducing the 
moving picture or the acoustic data may be data edited 
for simplified presentation or data for a partial scene pe- 
riod. 

[0475] The interface 63J is interfacing means with re- 
spect to a local area network (LAN) 70. Although a bus 
type LAN is used in Fig.30, several methods for connec- 
tion may be envisaged, such as ring connection or star 
connection. 

[0476] The bus 63K is a data input/output bus, exem- 
plified by an ISA bus, EISA bus or PCI bus. The oper- 
ating input unit 63L may be exemplified by a keyboard, 
a mouse, a pointing device, such as pad, pen or pointer,' 
a dial, having or not having a click function, and a push- 
button key. 

[0477] The display means 63M is a device used for 
demonstrating a picture, such as an input picture for the 
program ancillary data. A monitor device is routinely em- 
ployed. For household use, a picture of a television re- 
ceiver is used. In the case of a portable device, a small- 
sized lightweight type display device, such as a liquid 
crystal display device, is used. 

[0478] The program ancillary data generating device 
63, shown in Fig.30, may be formed by computer termi- 
nals, such as notebook or desktop personal computers, 
PDA, portable telephone sets, game machines, set top 



boxes, PC cards, domestic electrical utensils, VTR, tel- 
evision receivers, car navigation systems, wrist watches 
having the information processing functions, distribution 
terminals for music or games, financial terminals (ATM) 
5 etc., in addition to the so-called dedicated terminals. 
[0479] When the data creating device for program an- 
cillary data 63 is used in a broadcasting station or a pic- 
ture creating company, a database for program ancillary 
data 65, a database for commodities or services 66, a 
10 program data reproducing device 67, a device for cre- 
ating data for transmission 68 and a multiplexing 
processing device 69 are connected to the data creating 
device for program ancillary data 63 over LAN 70. 
[0480] The database for program ancillary data 65 is 
*s used not only for capturing and storing the program an- 
cillary data created in the primary inputting system, but 
also for storing the program ancillary data created anew 
in the aforementioned data creating device for program 
ancillary data 63. For capturing data from the primary 
20 inputting system, an interfacing equipment, not shown, 
is used. The program ancillary data is captured in the 
form of data transmission or of a recording medium. 
There is no limitation to the transmission system or to 
the recording medium configuration. 
25 [0481] The database for commodities or services 66 
is used for storing the detailed information pertinent to 
the commodities or services necessary for creating the 
program ancillary data. The information stored may be 
input separately by the secondary inputting system or 
so acquired from the aforementioned sponsorship transac- 
tion support system. If acquired from the sponsorship 
transaction support system, the sequence of the infor- 
mation is captured in the configuration of transmission 
or the recording medium. There is no limitation to the 
35 transmission system or to the recording medium conf iq- 
u rati on. 

[0482] The program data reproducing device 67 is 
means for reproducing the edited program data. As this 
reproducing means, such device that is in keeping with 
40 the medium having recorded the program data is used. 
If, for example, the program data is recorded on a mag- 
netic tape, a video tape recorder (VTR) is used, where- 
as, if the program data is recorded on a hard disc, a hard 
disc driving device is used. Of course, when the program 
45 data is recorded on an optical medium, an optical repro- 
ducing device is used. Meanwhile, the program data re- 
produced by the program data reproducing device 67, 
such as picture data or audio data, is used not only in 
the data creating device for program ancillary data 63 
50 but also for actual broadcast to the audience. 

[0483] The device for creating data for transmission 
68 is means for converting the program ancillary data 
into the data format matched to the outputting configu- 
ration for outputting the program ancillary data along 
55 with the picture data. An illustrative structure of the de- 
vice for creating data for transmission 68 is shown in 
Fig.31 . In the case of Fig.31 , the device for creating data 
for transmission 68 is made up by a LAN interface 68A, 
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a packet processing u n it 68 B for converting the pro gram 
ancillary data into packet data, a communication unit 
68C for control with outside, an access information ac- 
quisition unit 68D, a number of times of access imparting 
unit 68E and a bus 68F. 

[0484] In Fig.31 , the program ancillary data is convert- 
ed into packets of a data structure which is in meeting 
with the transmission format, using the packet process- 
ing unit 68B. Alternatively, the program ancillary data 
can be transmitted in the file transmission system. The 
access information acquisition unit 68D is means for ac- 
quiring the access information, as an index for the inter- 
est the audience has in the commodities or services per- 
tinent to each program ancillary data, from the database 
of the transaction brokerage business operator. For ac- 
quiring the information, the communication unit 68C is 
used. A communication unit which is in keeping with the 
communication network, used for connection to the ex- 
ternal database, is used forthe communication unit 68C. 
The communication may be made over a wired path or 
over a wireless path, without any limitations. 
[0485] The number of times of access imparting unit 
68E is means for according the results of sum of the 
number of times of access, on the commodity or service 
basis or on the scene basis, to the program ancillary da- 
ta. By providing such function in the device for creating 
data for transmission 68, the program ancillary data hav- 
ing the latest access information appended thereto may 
at all times be distributed in the broadcast or communi- 
cation configuration. The processing of summing the 
number of times of access events can be executed with- 
in the device for creating data for transmission 68 or may 
be directly received from the external database. 
[0486] This device for creating data for transmission 
68 forms a picture program data creating device 73 in 
conjunction with a multiplexing device 69, as later ex- 
plained. Meanwhile, the multiplexing device 69 is means 
for multiplexing the program ancillary data, packettzed 
in the device for creating data for transmission 68, into 
program data (packetized program data) supplied from 
the program data reproducing device 67. By this multi- 
plexing processing, picture program data of the ultimate 
output configuration is generated. 
[0487] Meanwhile, a gateway 71 is hardware or soft- 
ware means for interconnecting the Internet or the wire- 
less communication network with the LAN 70. The sys- 
tem of Fig.30 is used solely for transmitting the picture 
program data (output of the multiplexing device 69 or 
the program data for transmitting only the program an- 
cillary data. By and large, the transmission occurs in the 
transmission or communication configuration. 
[0488] A recording device 72 is means for recording 
picture program data on a recording medium for distri- 
bution to the consumer. Such a recording device which 
is in keeping with the recording medium is used as the 
recording device 72. The recording device of a large ca- 
pacity is routinely used. However, a recording device of 
a small capacity suffices for recording only the program 



ancillary data. 

[0489] The recording medium may be exemplified by 
a CD-ROM, a magnetic recording type medium, such 
as tape, floppy disc, hard disc or a magnetic card, an 
s optical recording medium, such as write-once CD, DVD 
or optical card, magneto-optical recording type medium, 
such as DVD, MD or MO, and a semiconductor memory, 
such as non-volatile memory. For recording, an electri- 
cal method of changing the magnetism or the light vol- 
ume depending on data for recording on a recording me- 
dium, or a method exploiting the semiconductor produc- 
ing technique, that is a method consisting in creating a 
master disc and a stamper, may be used. 
[0490] A transmitter 73 is means for transmitting or 
distributing to the audience the picture program data or 
the program ancillary data, over a wired route, such as 
optical cable, a power supply line or a telephone line, or 
a radio path, such as by surface waves or satellite 
waves, in either the broadcast configuration or in the 
communication configuration. 

[0491] Fig.30 shows am illustrative structure of a sys- 
tem in a routine system within a broadcasting station or 
within the picture creation business operator. Mean- 
while, the functions of the creation device for program 
ancillary data may be enclosed within the existing pic- 
ture program editing device. Fig.32 shows an illustrative 
structure of apicture program editing device fitted with 
a program ancillary data creating function. 
[0492] A picture program editing device 75, shown in 
Fig.32, includes a display controller 75A, a controller 
75B, a system controller 75C, a disc controller 75 D, a 
hard disc device 75 E, a special effect picture processing 
unit 75F, a picture input/output unit 75G, a video tape 
recorder (VTR) 75H, a speech input/output/mixer unit 
75 1, a program ancillary data processing unit 75 J, an op- 
erating input unit 75K and a display device 75L. 
[0493] In Fig.32, the are shown connected outside the 
main body unit of the device. However, the operating 
inputting unit 75K and the display means 75M may also 
be connected as one to the main body unit of the device, 
as in the case of a notebook personal computer, a PDA 
and a portable telephone set. Although the hard disc de- 
vice 75E and the video tape recorder 75H are shown 
enclosed in the main body unit, these may also be con- 
nected to outside the main body unit. For external con- 
nection, wired connection, either serial or parallel, may 
be used, in which case a coaxial cable, a twisted pair 
cable or an optical fiber may be used without any limi- 
tations. Alternatively, wireless connection may also be 
used without regard to the bandwidth. The spread spec- 
trum system, such as Bluetooth, may also be used. 
[0494] The display controller 75 A is means for repro- 
ducing and outputting picture data in signal forms (data 
forms) conforming to the display means 75M. Output 
signals may, for example, be a video output (NTSC vid- 
eo, composite video and S-video) and an RGB output. 
The display controller 75 A includes a data decoding 
function, such as one provided by MPEG (Moving pic- 
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ture Experts Group), depending on the purposes of use 
[0495] The controller 75B is means for controlling re- 
spective parts or commanding signal processing re- 
sponsive to commands input through the operating input 
unit 75L, and is routinely configured as a micro-proces- 
sor. The system controller 75C is means for controlling 
the write/readout control or signal processing for the vid- 
eo tape recorder 75H and, similarly to the controller 75B, 
is configured as a micro-processor. The functions of 
these controllers are prescribed by an operating pro- 
gram or by an application program. 
[0496] The disc controller 75D is means for controlling 
write/readout of the hard disc device 75E. In the hard 
disc device 75E, there is stored picture data or acoustic 
data being edited. The picture special effect processing 
unit 75F is means for applying optional special effects 
as specified by an editor, to picture data read out thro ugh 
the picture input/output unit 75G. For example, two-di- 
mensional effect processing, such as wiping, superim- 
posing, key synthesis, enlarging or contracting, or the 
three-dimensional effect processing, is earned out. The 
picture special effect processing unit 75F is routinely 
constructed in conjunction with a switcher. 
[0497] The picture input/output unit 75G is means for 
effecting picture data write/readout between it and plural 
video tape recorders 75H. Each video tape recorder 
75H has recorded therein picture program data as the 
picture material or the pre-edited picture program data. 
[0498] The speech input/output/mixer unit 75I is 
means for reading/writing acoustic data between it and 
the plural video tape recorders 75H and for mixing the 
acoustic data. Although a video tape is used in Fig.32 
as storage means for picture program data as editing 
material or pre-edited picture program data, an optical 
disc device or the like large capacity medium may also 
be used. 

[0499] The processing unit 75J for program ancillary 
data is means for implementing the functions of the 
aforementioned creating device 63 for program ancillary 
data. That is, the program ancillary data generating 
function or the index picture creating function etc. is car- 
ried out. The bus 75K is a data input/output bus and may 
for example, be an ISA bus, an EISA bus or a PCI bus! 
[0500] The operating input unit 75L may be exempli- 
fied by a keyboard, a mouse, a pointing device, such as 
pad, pen or pointer, a dial, having or not having a click 
function, and a pushbutton key. The display device 75M 
is a device used for displaying an input picture for pro- 
gram ancillary data and other pictures. A monitor device 
is routinely employed. For household use, a picture of 
a television receiver is used. In the case of a portable 
device, a small-sized lightweight type display device 
such as a liquid crystal display device, is used. 
[0501] A picture program editing device 75 may be 
formed by computer terminals, such as notebook or 
desktop personal computers, PDA, portable telephone 
sets, game machines, set top boxes, PC card, domestic 
electrical utensils, VTR, television receivers, car navi- 



gation systems, wrist watches having the information 
processing functions, distribution terminals for music or 
games, financial terminals (ATM) etc., in addition to the 
so-called dedicated terminals. 

5 

(b) Data creating picture 

[0502] A typical display of a creating picture, demon- 
strated on the display means 63M during creation of the 
™ program ancillary data, is hereinafter explained. 

(b-1) Typical display 1 



[0503] A first display picture 77 of Fig.33 is explained 
15 A display picture 77 of Fig.33is made up of a title bar 
77A, an input item display area 77B, in which the infor- 
mation relevant to the commodities or services to be in- 
put as the information pertinent to a scene to be created, 
and an already input information display area 77C for 
20 demonstrating the state of correlation of the commodity 
or service data with each scene along the time axis 
[0504] Meanwhile, Fig.33 is used for creating the pro- 
gram ancillary data concerning the commodities or serv- 
ices pertinent to the sponsorship. So, only a list of the 
25 commodities or services relevant to the sponsor is 
shown in the input item display area 77B. However, the 
information pertinent to commodities or services not in- 
cluded in the sponsors may also be displayed. 
[0505] In the input item display area 77B, there is 
30 demonstrated the specified information pertinent to the 
parties furnishing the commodities or services included 
in the sponsor and to the commodities or services fur- 
nished. The information pertinent to the commodities or 
services are provided with icons for facilitating the con- 
55 firmation on the picture. These icons are adapted to en- 
able discrimination of the corresponding commodities or 
services from the display configuration or color and from 
the letter information. In Fig.33, a symbol comprised of 
the combination of Roman letters representing the 
40 sponsor and numerical figures representing the com- 
modities or services furnished by the sponsor is allocat- 
ed to each icon. 

[0506] The already input information display area 77C 
is made up by a time code column 77C5, as time axis 
45 a picture column 77C1 , representing the content of each 
scene, an acoustic column 77C2, representing the ef- 
fect sound used in each scene or the content of the 
BGM, a data column 77C3, representing the information 
pertinent to the commodities or services input as pro- 
50 gram ancillary data, and a list column 77C4, providing 
a range of plural scenes displayed in a list in the same 
picture. In the display areas of these picture columns 
the index pictures of the respective scenes, icons (pic- 
turesque symbols) representing the content of the 
55 sound used in each scene, icons (picturesque symbols) 
representing the content of the commodities or services 
associated with each scene and the scope of the scenes 
of interest, represented by e.g., bar graphs, are dis- 
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played. 

[0507] It is noted that the index pictures represented 
in the picture column 77C1 may be created within the 
program ancillary data creating device 63, that is within 
the index picture creating unit 63G, or the index picture s 
used in the picture editing device 75 may be used. 
[0508] There are several methods of allocating the in- 
formation pertinent to the commodities or services to the 
data column 77C3. One such method is to drag and drop 
the icon selected in an input item display area 77B. In 
this case, the operator may drop the selected icon on 
an index picture of a corresponding scene, or in a data 
column of the corresponding scene. Without regard to 
the input method used, the picturesque symbols, indi- 
cating the content of the commodities or services input, 
are displayed in the relevant positions of the data col- 
umn 77C3. 

[0509] Meanwhile, if the information on the level of the 
name of the sponsor has already been finished in the 
primary input system (the inputting of the Roman letters 
[A] or [B] has already been finished in the case of Fig. 
33), the display on the picture is replaced by the dropped 
content by the aforementioned drag and drop operation. 
For example, the display of [A] is replaced by display [A- 
1). 

[0510] The inputting method is not limited to the drag 
and drop system. For example, the inputting to the data 
column 77C3 may be made in such a manner that se- 
lection of an icon comprised only of the name of the 
sponsor displayed in the data column 77C3, such as by 
clicking, causes the display of the candidate for selec- 
tion in the input item display area 77B to be changed, 
for example, to flicker or be changed in color, with the 
inputting in the data column 77C3 being finished by the 
operator specifying one of the candidates for selection. 
[0511] Since there is a limit to the area that can be 
displayed, there are a number of occasions where only 
one commodity or service is displayed, even although 
the information pertinent to plural commodities or serv- 
ices is associated with one scene, as in the case of the 
display shown in Fig. 33. Of course, there may be occa- 
sions where registration of the information pertinent to 
plural commodities or services may be noticed from the 
display methods. For example, the display content may 
be confirmed by parallel display of plural icons or over- 
lapped display. 

(b- 2) Typical display 2 

[051 2] However, if the icons are arrayed in parallel, or 
if the display area is shifted for overlapped display, it is 
unavoidable that the content become difficult to confirm. 
So, a second display picture 78 shown in Fig. 34 is pro- 
vided. In the display picture 78 shown in Fig.34, an en- 
larged display window 77D of the data column 77C3 is 
indicated overlapped on the aforementioned first display 
picture 77. 

[0513] Fig.34 shows an instance where the number 



of the commodities or services associated with a sole 
scene is approximately three (data columns in the draw- 
ing being 77C10, 77C20 and 77C30). Of course, there 
are cases where three or more commodities or services 
are associated with each scene, in which case the por- 
tions not displayed in the display area may be adapted 
to be confirmed through the operation of a scroll bar pro- 
vided on a picture, or the information displayed on the 
enlarged display window 77D may be limited to the con- 
tent of the data column 77C3 to enable more information 
to be displayed. 

[0514] Meanwhile, there are also occasions where it 
is desirable to confirm, at one glance, the information 
concerning the same commodities or services appear- 
ing at plural scenes in actual editing operations. If, in 
this case, the controller 63D is caused to execute such 
a function in which, when an operator performs an op- 
eration of specifying the commodities or services of the 
input item display area 77B as he or she depresses an 
operating key associated with the function in question, 
display of the relevant information in the data column 
77C3 is changed, for example, the display flickers or is 
changed in color, the editing operation can be improved 
in efficiency. As the method of implementing a similar 
function, it may be envisaged to reverse the operating 
sequence, for example, to initially specify certain com- 
modities or services of the input item display area 77B 
and to then depress an operating key to which the func- 
tion in question is allocated. This function can be exploit- 
ed even on the first display picture 77. 

(b-3) Typical display 3 

[0515] By providing the above-described two display 
pictures, the operator engaged in an editing operation 
is able to confirm which information pertinent to com- 
modities or services is associated with which scene. 
Meanwhile, it should be possible how such information 
is presented to the audience, otherwise the system 
would be practically unusable. 

[0516] So, a third display picture 79, shown in Fig.35, 
is provided. This picture indicates how the information 
pertinent to plural commodities or services associated 
with a given scene is to be demonstrated. That is, this 
picture is a display example for a meta-pocket in the 
present specification. This picture will be explained in 
detail later in connection with the explanation of the au- 
dience side system. In Fig.35, only schematics are ex- 
plained. 

[0517] In the display picture 79, there are displayed 
an information display column 79A indicating the iden- 
tification code information of the program ancillary data 
and the detailed information on the commodities or serv- 
ices, jump buttons 79B to 79D in which is embedded the 
link information to other display pictures associated with 
the display picture 79, and an exit button 79E to which 
is associated the operation of closing the display picture. 
[051 8] The display content of the information display 
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column 79A are operatively linked with the input results 
in the first and second display pictures. In case the dis- 
play on the information display column 79A is inappro- 
priate, the display position on the picture is corrected. 
The information in question is ultimately output as the s 
program ancillary data. The so generated program an- 
cillary data is recorded in the program ancillary data re- 
cording unit 63C. 

[051 9] Meanwhile, a meta-cart 79B, among the jump 
buttons, is a button used for jumping to a picture for con- 10 
firming the information pertinent to the commodity the 
audience desires to purchase and which has not been 
definitely transmitted to the brokerage business opera- 
tor. Since there is no other picture in the editing stage 
than a template picture provided from the outset, no ex- is 
planation is given on an exemplary picture. 



(b-4) typical display 4 

[0520] Fig.36 shows a fourth display picture 80 This 20 
display picture 80, which is a display example of a pic- 
ture demonstrated in case a meta-mall 79C as one of 
the jump buttons shown in Fig.35 is depressed, corre- 
sponds to the content of the list column 77C4 of Fig.33. 
[0521 ] Fig.36 shows a typical picture 80 in case meta- 25 
pockets of three different scenes are to be recognizable 
on a sole picture. In Fig.36, index pictures are displayed 
in association with each scene to permit facilitated rec- 
ognition of each scene. In the display picture, since the 
display area 80A that can be allocated to one scene be- so 
comes narrower than with the meta-pocket, only several 
information of each meta-pocket are selectively demon- 
strated. As a method for selective display, it may be con- 
templated to demonstrate a preset number of informa- 
tion from the leading end of each meta-pocket. 35 

(b-5) Typical display 5 

[0522] Fig.37 shows a fifth display picture 81 . This dis- 
play picture is a display example of a picture demon- 40 
strated when a commodity information site 79D, among 
the jump buttons shown in Fig.35, is depressed. This 
display picture is used for confirmation for such a case 
where a display picture of the information concerning 
the individual commodities or services is provided at the 45 
data creating device for program ancillary data. 
[0523] However, in case the information is on a home 
page of the sponsor business operator or on a home 
page of a brokerage business operator, it is generally 
unnecessary to confirm the content of the information, so 
and hence the function in question is unneeded, al- 
though the home page in question may be optionally ac- 
cessed to confirm its content. 



(c) processing operations 

[0524] Finally, the processing operations in the editing 
operation employing the program ancillary data creating 
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device 63 are explained. 

[0525] Fig.38 shows an operating sequence under a 
condition in which no information has been registered 
in the data column 77C3 in the already input information 
display area 77C of Fig.33. The controller 63D reads out 
the layout information corresponding to the display ex- 
ample 1 (Fig.35) from the layout information holding unit 
63H. The controller 63D then causes the information 
concerning the sponsorship-related goods or services, 
pertinent to the picture program in question, as read out 
from the database for commodities or services 66, to be 
displayed in the input item display area 77B. Simultane- 
ously, the controller 63D demonstrates the index pic- 
ture, as read out from the picture editing device or the 
index picture creating unit 63G, in association with the 
corresponding scene (step S41). 
[0526] In this state, the controller 63D verifies whether 
or not an icon representing the commodities or services 
demonstrated in the input item display area 77B has 
been selected (step S42). This decision operation is re- 
peated as long as negative results persist. When the op- 
erator makes selection and affirmative results are ob- 
tained, the controller 63D causes the corresponding in- 
formation to be displayed in the data column 77C3 cor- 
responding to the index picture to which the selected 
icon has been dragged and dropped (picture column 
77C1). Although the icon drop destination is the index 
picture display column, ft may also be the data column 
77C3. 

[0527] The next operation is carried out in accordance 
with Fig.39. The controller 63D reads out the layout in- 
formation corresponding to the display example 1 (Fig. 
35) from the layout information holding unit 63H, while 
displaying on the input item display area 77B the infor- 
mation concerning the sponsorship-related commodi- 
ties or services pertinent to the picture program as read 
out from the database for commodities or services 66 
The controller 63D also displays the index picture, read 
out from the picture editing device or the index picture 
creating unit63G, in association with the corresponding 
scene, while reading out the previously input information 
concerning the commodities or services from the input 
data recording unit 63B to display the so read out infor- 
mation in the data column 77C3 (step S45). 
[0528] In this state, the controller 63D verifies whether 
or not the icon representing the commodities or services 
displayed in the input item display area 77B has been 
selected (step S46). This decision operation is repeated 
as long as negative results prevail. When the operator 
makes selection and affirmative results are obtained, 
the controller 63D causes the corresponding informa- 
tion to be displayed in the data column 77C3 corre- 
sponding to the index picture to which the selected icon 
has been dragged and dropped. Although the icon drop 
destination is the data column 77C3, it may also be the 
index column (picture column 77C1 ), as in the example 
of Fig.38. 

[0529] By repeating the operation of Fig.39, the infor- 
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mation concerning the commodities or services making 
up the program ancillary data is input. 

(C-4) summary 

[0530] By constructing the above-described creating 
system for program ancillary data, the inputting and the 
editing of the voluminous information on the commodi- 
ties or services indispensable for the realization of data 
broadcast or distribution attractive to the audience can 
be realized efficiently. In particular, since this enables 
effective utilization of the information concerning the 
commodities or services accumulated in arriving at the 
sponsorship tie up for commodities or services for pic- 
ture programs, the operating efficiency is improved not 
only on the working site for program creation but also 
on the working site for editing. 

(D) Digital Picture Program Receiving Reproducing 
System 

[0531] The digital picture program receiving system 
may be roughly divided into an audience side system 
receiving or reproducing the digital picture program and 
an brokerage business operator side system which links 
the interest on the part of the audience concerning the 
commodities or services as imaged objects to the infor- 
mation fumishment and commercial transactions. 
These respective systems are hereinafter explained. 

(D-1 ) Program ancillary data processing device 

(a) Terminal structure 

[0532] The program ancillary data processing device, 
forming the audience side system, is classified into a 
device directly receiving or reproducing a digital picture 
program, referred to below as a primary terminal, and a 
device receiving the necessary information transferred 
from the primary terminal, referred to below as a sec- 
ondary terminal. Meanwhile, the method for acquiring 
the digital picture program includes a method for receiv- 
ing the digital picture program through ground or satel- 
lite waves, that is over a radio path, a method for receiv- 
ing the digital picture program through a cabled network, 
such as telephone line, inclusive of ISDN, an optical fib- 
er network or a cable television network, and a method 
for reproducing the digital picture program from a re- 
cording medium. 

[0533] Meanwhile, the program ancillary data 
processing device may also be a hybrid device, also 
having the function of demodulating or decoding pro- 
gram data and the outputting function, instead of being 
dedicated to the processing of the program ancillary da- 
ta. 



(a-1) Primary terminal 

[0534] Figs.40 to 43 show four embodiments of the 
primary terminal. In any of these cases, a reproducing 
device 85M, an operating input device 85N and display 
means 850 are shown connected outside of the main 
body unit of the device, these may also be formed as 
one with or enclosed within the main body unit. Con- 
versely, respective functional units or recording units 
may also be connected outside of the main body unit of 
the device. Needless to say, the connection between the 
main body unit and the external device may be wired or 
wireless. As for the method for communication or trans- 
mission between the devices, specified connection sys- 
tems may be used. 

[0535] Referring first to Fig. 40, a processing device 
85 for program ancillary data includes a display control- 
ler 85 A, an number of times of access calculating unit 
85B, an input data recording unit 85C, a program ancil- 
lary data recording unit 85D, an extraction unit for pro- 
gram ancillary data 85E, a controller 85F, a receiving 
unit 85G, an index picture recording unit 85H, a com- 
munication unit 85 1, an index picture creation unit 85 J, 
a commodity/service information presence/absence de- 
cision unit 85K, a bus 85L, a reproducing device 85M, 
an operating input unit 85N and display means 850. 
[0536] The display controller 85A is means for repro- 
ducing and outputting received or reproduced picture 
data in signal forms (data forms) conforming to the dis- 
play means 850. Output signals may, for example, be 
a video output (NTSC video, composite video and S- 
video) and an RGB output. The display controller 85A 
includes a data decoding function, such as one provided 
by MPEG (Moving Picture Experts Group), depending 
on the usage and application. 

[0537] The received wave may be in the broadcast 
form or in the communication form. In any of the trans- 
mission configurations, the propagation path of the re- 
ceived wave includes a radio path, such as ground or 
satellite waves, and a cabled path, such as a telephone 
line, a data line and an optical fiber. 
[0538] The number of access times calculating unit 
85 B is means for calculating the number of times of ac- 
cess events on the scene basis, based on the number 
of times of access events on the commodity or service 
basis. This unit is unnecessary for a terminal not having 
or not in need of the function of displaying the number 
of times of access events on the scene basis. The unit 
is similarly unnecessary for such a case where the 
number of times of scene-based access events can be 
acquired as reproduced data or as received data inclu- 
sive of data received independently of the digital picture 
program. 

[0539] The number of access times calculating unit 
85B calculates the sum of the number of times of access 
events to the commodities or services appearing as im- 
aged objects in each scene, usually only the commodi- 
ties or services appearing in the program ancillary data, 
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in terms of the number of times of the access events to 
the scene in question, without regard to the route of ac- 
quisition of the information on the number of individual 
access events pertinent to the commodities or services. 
The calculated results are used when displaying the 
content of the program ancillary data on the picture such 
as when displaying meta-pockets or meta-malls. That 
is, even when the information on the scene-based ac- 
cess events is not included in the program ancillary data 
received or reproduced, the number of times of access 
events representing the scene-based interest enter- 
tained by the audience can be correlatively displayed. 
[0540] As for the acquisition route, the information 
may be acquired as picture program data, that is as pro- 
gram ancillary data along with the program data, as pro- 
gram ancillary data by itself or as the number of times 
of access events by itself. The transmission medium 
may be wired or wireless. The information may also be 
acquired as replay data from the recording medium. 
[0541] The input data recording unit 85C is a record- 
ing or storage medium used for recording data input 
through the operating input unit 85N for communication 
purposes. These mediums may be exemplified by a 
CD-ROM, a magnetic recording type medium, such as 
tape, floppy disc, hard disc or a magnetic card, an optical 
recording medium, such as write-once CD, DVD or op- 
tical card, magneto-optical recording type medium, such 
as DVD, MDorMO, and a semiconductor memory, such 
as non-volatile memory or RAM. Of these, a RAM is felt 
to be routinely used. 

[0542] It is unnecessary for the input data recording 
unit 85C to be a recording unit dedicated to recording 
the input data. For example, it is naturally possible to 
use the physically same medium for the input data re- 
cording unit 85C as the recording unit 85D for program 
ancillary data, if only the recording area used is a differ- 
ent area. Meanwhile, the input data recording unit 85B 
may be provided within the display controller 85A. 
[0543] The recording unit 85D for program ancillary 
data is a recording or storage medium for recording the 
program ancillary data acquired as received or repro- 
duced data. The medium used is routinely a RAM, how- 
ever, as in the case of the input data recording unit 85C, 
there is no particular limitation to the sort of the medium 
provided that it allows for writing program ancillary data. 
There is also no necessity for the recording unit 85D for 
program ancillary data to be a recorder dedicated to re- 
cording the program ancillary data. For example, the 
medium which is physically the same as the aforemen- 
tioned input data recording unit 85C may be commonly 
used. 

[0544] The extraction unit for program ancillary data 
85E is means for separating and extracting the program 
ancillary data from the received or reproduced picture 
program data. Fig.20 shows typical program ancillary 
data. This function may be implemented in dedicated 
hardware or as software. 

[0545] The controller 85F is means for controlling var- 



ious components of the processing device for program 
ancillary data 85 and for executing signal processing, 
and is generally configured as a micro-computer. The 
function of the controller 20F is routinely prescribed by 
5 the operation program and by the application program. 
[0546] The receiving unit 85G is a unit for receiving 
picture program data or program ancillary data transmit- 
ted over a wired or wireless path, and executes preset 
demodulation and decoding. 
10 [0547] The index picture recording unit 85H is a re- 
cording or storage medium for recording an index pic- 
ture extracted from the program ancillary data, an index 
picture received from outside over a communication 
path, for example, from a brokerage database acting as 
'5 an intermediary for commercial transactions, or a pic- 
ture for recording an index picture created in the pro- 
gram ancillary data processing device 85. 
[0548] These mediums may be exemplified by a 
CD-ROM, a magnetic recording type medium, such as 
20 tape, floppy disc, hard disc or a magnetic card, an optical 
recording medium, such as write-once CD, DVD or op- 
tical card, magneto-optical recording type medium, such 
as DVD, MD or MO, and semiconductor memory, such 
as non-volatile memory or RAM. However, a RAM is felt 
25 to be routinely used. 

[0549] The communication unit 85! is means for com- 
munication with an external terminal over a network, 
which may be wired or wireless. In case a bidirectional 
path is used for receiving program ancillary data or pic- 
30 ture program data, the receiving unit 85G and the com- 
munication unit 85I may be used in common. 
[0550] The communication unit 85! is used in case the 
audience feels that the content of the program ancillary 
data displayed on the display picture is insufficient and 
35 accesses an external database, that is downloads the 
more detailed information concerning the commodities 
or services as imaged objects, or in case the intention 
to purchase the commodities or services as imaged ob- 
jects is to be transmitted. It is also used for individually 
40 downloading the program ancillary data. 

[0551] The index picture creation unit 85J is means 
for creating an index picture based on a representative 
frame picture used for confirming the content of the pro- 
gram ancillary data. Since the index picture is used for 
^ confirming a corresponding scene, it is generated as a 
picture the picture information of which is reduced in pic- 
ture quality as compared to the picture used for real 
broadcast. It is, however, not mandatory to lower the pic- 
ture quality such that the index picture may be of the 
so same information as that of the actual broadcast. 

[0552] Usually, a frame picture representative of the 
content of each scene is used as an index picture. Ex- 
amples of the index picture include a leading frame of a 
corresponding scene, a frame which permits facilitated 
55 recognition of the sponsorship-related commodities or 
services, as selected by the audience, and a trailing end 
frame of the corresponding scene. If the rule for select- 
ing the frame picture in this manner is set, the index pic- 



52 




103 

ture creation unit 85J is able to create the index picture 
automatically. However, if the selection by the audience 
is resorted to, the frame picture at the time point (time 
code) as specified by the operating input unit 85N is 
used. Additionally, the index picture may be represented s 
with computer graphics. 

[0553] Meanwhile, in case the index picture can be 
acquired by itself or as a part of the program ancillary 
data, there is no necessity for the index picture creation 
unit 85J to be provided in the device. However, if a time 
point scene used by the audience as an index picture is 
to be arbitrarily selectable, there is meaning in providing 
the index picture creation unit 85J. 
[0554] The commodity/service information presence/ 
absence decision unit 85K is means which, in case there 
is embedded the identification information specifying 
whether or not the information on the commodities or 
services is contained in the program ancillary data, 
renders it possible to discriminate, without the necessity 
of analyzing the data content, whether or not the pro- 
gram ancillary data being processed has to do with com- 
modities or services. 

[0555] If this decision unit is used, the program ancil- 
lary data containing the information pertinent to com- 
modities or services can be distinguished from other 
program ancillary data with only a small signal process- 
ing volume. Moreover, only the program ancillary data 
containing the information pertinent to the commodities 
or services can be selectively sent to the display con- 
troller 85A for display on the display picture or storage 
in a recording medium. 

[0556] Moreover, if, in case of creating an index pic- 
ture within a program ancillary data creating device, 
there is not contained the information pertinent to com- 
modities or services, there is no necessity for confirming 
the commodities or services on the index picture, so that 
the creation of the index picture may be dispensed with. 
However, this applies only for a case where the index 
picture is used exclusively for confirming the commodi- 
ties or services, such that, if the index picture is used 
for other purposes, creation of the index picture may be 
necessary. 

[0557]. Meanwhile, should the information pertinent to 
the subject-matter of the sponsorship be included in the 
program ancillary data, a sponsor information presence/ 
absence decision unit, similar to the commodity/service 
information presence/absence decision unit 85K, may 
be provided for verifying, from the relevant information 
in the program ancillary data, whether or not there is 
contained the information concerning the subject-matter 
of the sponsorship. 

[0558] These decision units are particularly effective 
when a display picture (m eta-pocket in the present 
specification) used for collecting the information perti- 
nent to the commodities or services appearing as an im- 
aged object in each scene in the aforementioned sup- 
plementary information data creating system into one is 
not prepared. 



[0559] For example, such a use becomes possible in 
which, if the component units of the program ancillary 
data contained in the received or reproduced signal are 
commodities or services, the program ancillary data rel- 
evant to the same scene is first extracted, using the in- 
formation specifying each scene, such as UMID or time 
code, as a retrieving key, and in which the data pertinent 
to commodities or services or the data pertinent to the 
sponsorship is then sorted and extracted, using the 
aforementioned decision units, with the m eta-pocket be- 
ing then created from the sorted information pertinent to 
the commodities or services. 

[0560] The bus 85L is a data input/output bus, which 
may, for example, be an ISA, EISA pr PCI bus. The re- 
producing device 85M is means for reproducing the pic- 
ture program data or program ancillary data distributed 
as recording medium. For the reproducing device 85M, 
such a device which is in keeping with the recording me- 
dium having picture program data or the program ancil- 
lary data, recorded thereon, is preferably used. Since a 
large capacity medium is required for program data, 
DVD or MO is presupposed as medium. For program 
ancillary data, a data volume smaller than that for picture 
program data suffices, and hence a non-volatile mem- 
ory or a CD-ROM etc. is presupposed in addition to the 
above mediums. However, the possible storage capac- 
ity is changed appreciably with the progress in technol- 
ogy and the debut of a new medium needs to be taken 
into account. Consequently, there is no limitation as to 
the name or the compatibility relationships of the stor- 
age mediums used. The same holds for other mediums. 
[0561] The operating input unit 85N may be enumer- 
ated by, for example, a keyboard, a mouse, a pointing 
device, such as pad, pen or pointer, a dial, having or not 
having a click function, and a pushbutton key. A remote 
controller may also be used. 

[0562] The display means 850 is a device used for 
confirming the content of the program ancillary data. In 
general, such display means which is also capable of 
displaying a picture program is used. In general, a dis- 
play picture of a television receiver or a monitor device 
may also be used. In the case of a portable device, a 
liquid crystal device or the like small-sized lightweight 
display device is used. 

[0563] The terminal operating as 'program ancillary 
data processing device 1 shown in Fig. 40 includes, in ad- 
dition to dedicated devices for processing program an- 
cillary data, computer terminals, such as notebook or 
desktop personal computers, portable information ter- 
minals (PDA), portable telephone sets, game machines, 
set top box, PC card, domestic electrical utensils, VTR, 
television receivers, car navigation systems, wrist 
watches having the information processing functions, 
and remote controllers. The same holds for the program 
ancillary data processing devices hereinafter explained. 
[0564] Reference is now made to Fig.41 in which the 
parts or components same as those of Fig.40 are indi- 
cated by the same reference numerals. Fig.41 differs 
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from Fig.40 in that a selection scene position recording 
unit 85P is substituted for the commodity/service infor- 
mation presence/absence decision unit 85K. The selec- 
tion scene position recording unit 85P is a recording or 
storage medium for the audience to record the informa- 
tion of the scene position selected as the display posi- 
tion of the program ancillary data. Usually, a RAM is 
used for this recording unit. However, other mediums 
may also be used. 

[0565] By providing the function in question, selection 
of the scene position displayed as an index picture may 
be up to the discretion of the audience. In particular, 
since the audience may be interested in a wide variety 
of commodities or services, the index picture provided 
by the picture producer is not necessarily suitable to the 
commodities or services in which the audience is inter- 
ested. By providing the above function, the scene posi- 
tion that can be readily recognized by the audience may 
be adopted as the index picture. In case the selecting 
function on the part of the audience is to be displayed 
in creating the index picture, the selection scene posi- 
tion recording unit 85P needs to be provided in the case 
of Fig.40. 

[0566] Reference is now made to Fig.42 in which the 
parts or components same as those of Fig.40 are indi- 
cated by the same reference numerals. Fig.42 differs 
from Fig.40 in that an index picture extraction unit 85Q 
is substituted for the index picture creation unit 85J and 
in that a meta-pocket/meta-mail creation unit 85R is 
substituted for the commodity/service information pres- 
ence/absence decision unit 85K. 
[0567] The index picture extraction unit 85Q extracts 
an index picture when one is contained in the received 
data or reproduced data, without regard to whether the 
index picture is contained as a part of the program an- 
cillary data or is contained by itself. 
[0568] The meta-pocket/m eta-mall creation unit 85R 
is means for creating, in case the meta-pocket or the 
meta-mall is not included in the program ancillary data, 
the corresponding information, based on the program 
ancillary data received or reproduced. The meta-pocket 
is the program ancillary data which corresponds to the 
information pertinent to commodities or services ap- 
pearing in a specified scene and which routinely in- 
cludes the layout information, whereas the meta-mall is 
the program ancillary data which is adapted for enabling 
plural meta-pockets associated with respective plural 
scenes to be viewed at a glance in one picture, and 
which routinely includes the layout information. 
[0569] Meanwhile, the meta-mall can be created from 
the metadata concerning plural scenes, as determined 
under a specifying rule or subject to designation by the 
audience, or can be created from the information perti- 
nent to individual commodities or services. 
[0570] Reference is now made to Fig.43 in which the 
parts or components same as those of Fig.40 are indi- 
cated by the same reference numerals. Fig.43 differs 
from Fig.40 in that an external data presence/absence 



detection unit 85Q is substituted for three units, namely 
the index picture recording unit 85H, index picture cre- 
ation unit 85J and the commodity/service information 
presence/absence decision unit 85K. 

s [0571 ] The external data presence/absence detection 
unit 85Q is means effective when data (flags) indicating 
the external presence of the information pertinent to the 
corresponding scene in the program ancillary data are 
used. This is effective when the data volume of the pro- 

10 gram ancillary data is to be reduced in data volume or 
when services are to be rendered to members. That is, 
since the information on the commodities or services 
significant for the audience is not contained in the pro- 
gram ancillary data received or reproduced, the data 

is volume is appreciably reduced, while the information 
may be adapted for not being furnished to the audience 
except specified members. 

[0572] If the presence of the external data is detected 
in the external data presence/absence detection unit 
20 85Q, the external database etc. is accessed through the 
communication unit 851 to download the necessary in- 
formation. Meanwhile, the information concerning the 
accessed site may be pre-registered, or recorded in the 
program ancillary data. If services are rendered to spec- 
ks ified members, authentication means, not shown, are 
used. 

(a-2) Secondary terminal 



30 [0573] Fig.44shows an embodiment of the secondary 
terminal. Although Fig.44 shows a case where any func- 
tional units are enclosed or unified in a main body unit 
of the device, these may be externally connected to the 
main body unit. The connection between the main body 
35 unit and the external device may be wired or wireless. 
As for the method for communication or transmission 
between the devices, specified connection systems 
may be used. 

[0574] Meanwhile, a typical structure of the second- 
^0 ary terminal is shown as in Fig.44 because a portable 
type terminal is presupposed as a secondary terminal. 
The secondary terminal may, of course, be a standstill 
device. The typical structure is convenient when the pri- 
mary and secondary terminals are differentially used, as 
45 when the primary terminal is used in the user's premises 
and the secondary terminal is used outdoors. In this con- 
sideration, Fig.44 shows an example of acquiring the 
necessary information by wireless communication with 
the primary terminal (program ancillary data processing 
50 device) 85, such as television receiver. Of course, wired 
connection may be used as the connection between the 
secondary and primary terminals. 
[0575] Fig.44 shows a program ancillary data 
processing device 87 including a display controller 87A, 
55 a program ancillary data recording unit 87B, an input da- 
ta recording unit 87C, a controller 87D, a communication 
unit (downloading unit) 87E, a bus 87F, an operating in- 
put unit 87G and display means 87H. Although not 
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shown in Fig.44, data used in the secondary terminal 
may be acquired from the primary terminal in the form 
of a recording medium. In this case, a reproducing de- 
vice is required. 

[0576] The display controller 87A is means for repro- 
ducing and outputting received or reproduced picture 
data in signal forms (data forms) conforming to the dis- 
play means 87H. Output signals may, for example, be a 
video output (NTSC video, composite video and S-vid- 
eo) and an RGB output. The display controller 87A in- 
cludes a data decoding function, such as one provided 
by MPEG (Moving Picture Experts Group), depending 
on the usage and application. 

[0577] The program ancillary data recording unit 87B 
is a recording or storage medium for recording the pro- 
gram ancillary data received and reproduced. These re- 
cording mediums include a CD-ROM, a magnetic re- 
cording type medium, such as tape, floppy disc, hard 
disc or a magnetic card, an optical recording medium, 
such as write-once CD, DVD or optical card, magneto- 
optical recording type medium, such as DVD, MD or 
MO, and a semiconductor memory, such as non-volatile 
memory or RAM. Of these, the RAM is felt to be routinely 
used. 

[0578] It is unnecessary for the program ancillary data 
recording unit 87B to be a recording unit dedicated to 
recording the input data. For example, it is naturally pos- 
sible to use the physically same medium for the input 
data recording unit 87C as the data recording unit for 
program ancillary data 87D if only the recording area 
used is a different area. 

[0579] The input data recording unit 87C is a record- 
ing or storage medium for recording data input through 
the operating input unit 87G for communication purpos- 
es. The medium used is routinely a RAM. However, as 
in the case of the program ancillary data recording unit 
87B, there is no particular limitation to the sort of the 
medium provided that it allows for writing input data. It 
is unnecessary for the input data recording unit 87C to 
be a recording unit dedicated to the recording of the in- 
put data. On the other hand, the input data recording 
unit 87C maybe provided within the display controller 
87A. 

[0580] The controller 87D is means for controlling var- 
ious components making up the program ancillary data 
processing device 87 and for carrying out signal 
processing, and is configured as a micro-computer. 
Usually, the function of the controller 87D is prescribed 
by the operation system and by the application program. 
[0581] The communication unit87E is means for com- 
munication with outside, especially with the primary ter- 
minal 85, to acquire the necessary information. The 
communication path may be wired or wireless, as afore- 
said. The same holds for the case of communication 
with an external equipment other than the primary ter- 
minal. 

[0582] The bus 87E is an input/output bus for data, 
such as an ISA bus, an EISA bus or a PCI bus. The 



operating input unit 87G may, for example, be a key- 
board, a mouse, a pointing device, such as pad, pen or 
pointer, a dial, having or not having a click function, and 
a pushbutton key. A remote controller may also be used. 
5 [0583] The display means 87H is a device used for 
confirming the content of the program ancillary data. In 
the case of a portable device, a small-sized lightweight 
display device, such as a liquid crystal device, may be 
used. In the case of a standstill type device, a monitor 
device or a television receiver may be used. 
[0584] The terminal operating as the 'program ancil- 
lary data processing device* shown in Fig.44 includes, 
in addition to dedicated devices for processing program 
ancillary data, computer terminals, such as notebook or 
desktop personal computers, portable information ter- 
minals (PDA), portable telephone sets, game machines, 
set top boxes, PC cards, domestic electrical utensils, 
VTR, television receivers, car navigation systems, wrist 
watches having the information processing functions, 
and remote controllers. 15. 

(b) Display picture example 

[0585] Next, display picture examples of the program 
ancillary data processing devices 85, 87 are explained. 

(b-1) Presence notice picture of program ancillary data 

[0586] Figs.45 and 46 show a picture display example 
in a scene where the program ancillary data pertinent to 
the commodities or services, such as meta-mall or me- 
ta-pocket (list file), are associated with the picture pro- 
gram (real broadcast). It is noted that this picture display 
is made when the display function notifying to the audi- 
ence that the program ancillary data pertinent to the 
commodities or services is so associated is set to the 
[ON state] by the audience. If conversely the audience 
sets the function to the [OFF state], display of Figs.45 
and 46 is not made. 

[0587] Fig.45 shows a display example in case of di- 
visionally demonstrating the display area into a real 
broadcast area 90 and an auxiliary information area 91 . 
In the case of Fig.45, a notice symbol 91 A of a scene, 
with which the program ancillary data pertinent to the 
commodities or services is associated, is demonstrated 
in the auxiliary information area 91 . On the other hand, 
Fig. 46 shows a display example in case of using the en- 
tire display area for the real broadcast area 90. In Fig. 
46, the notice symbol 91 A for the scene with which the 
program ancillary data is associated is displayed super- 
imposed on the real broadcast picture. It is similarly pos- 
sible in the case of Fig.45 to display the symbol in su- 
perimposition on the real broadcast picture. 
[0588] There may also be envisaged a method of 
changing the display of an indicator provided on a front 
panel of an electronic equipment, such as a remote con- 
troller or a television receiver, such as by lighting, flicker 
or change of display color, or by producing an alarm 
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sound. 



(b-2) Display example of meta-pocket 

(1) Meta-pocket display example 

[0589] Figs.47 to 50 show a display picture example 
of the information pertinent to the commodities or serv- 
ices appearing in each scene, that is a display example 
of a meta-pocket, as is so termed in the present speci- 
fication. It is noted that the meta-pocket may be received 
or reproduced as program ancillary data or may be cre- 
ated by signal processing in the program ancillary data 
processing device. 

[0590] Figs.47 to 49 shows examples of display pic- 
tures having the same layout structure. Meanwhile, Fig 
47 is directed to a display example of a motion picture 
drama or a variety program, Fig.48 is directed to a dis- 
play example of a sports program and Fig.49 is directed 
to a display example of the information pertinent to the 
music or pictures used in the picture program. In these 
figures, the content conforming to the object is displayed 
as the information on the commodities or services. 
[0591] First, the layout structure common to these dis- 
play pictures 93 is explained with reference to Fig.47 
These display pictures each include an identification 
number display column 93A, as meta-pocket manage- 
ment information, a scene-based access information 
display column 93B, a display column forthe information 
pertinent to commodities or services 93C, an access in- 
formation display column on the commodity or service 
basis 93D, a meta-cart button jump button 93E embed- 
ding the link information to the meta-cart, a meta-mall 
jump button 93F, embedding the link information to the 
meta-mall, a commodity information site jump button 
93G, embedding the link information to the detailed in- 
formation pertinent to commodities or services as se- 
lected on the picture, a purchase button 93H used for 
inputting the intention to purchase and an end button 
93I for finishing the meta-pocket display. The scene- 
based access information display column 93B sums the 
access states from the audience to the commodities or 
services appearing in the meta-pocket, on the scene ba- 
sis. The access information display column on the com- 
modity or service basis 93D shows the accessing infor- 
mation from the audience to the commodities or servic- 
es appearing in the meta-pocket in question. 
[0592] In the case of Fig.47, the audience is able to 
acquire, at a time, the information pertinent to plural ob- 
jects appearing in an associated scene, for example, the 
information pertinent to the commodities or services 
such as performer's apparel, hair making, sunglasses' 
tables, sofas, location equipment etc. These information 
are relevant to the apparel or small articles, worn by the 
performers, or the fitting used in e.g., a studio, in dis- 
tinction from the information pertinent to the commodi- 
ties or services for so-called television shopping, and 
hence have not so far been taken into account in con- 



nection with information furnishment. 
[0593] Since these information are displayed, the 
business operator which furnished the commodities or 
services are given the opportunity of new advertisement 
s and hence the opportunity for latent business transac- 
tions. On the other hand, the audience is given the in- 
formation pertinent to the commodities or services in 
which the audience is interested, rather than the infor- 
mation of insipid uniformity. 
io [0594] In the case of Fig.47, only the classification of 
the commodity items, names of commodity, name of the 
sponsor business operator and the URL, are indicated 
in the display coiumn for the information pertinent to 
commodities orservices 93C, to display in frontthe func- 
w tion as a platform for acquiring the information pertinent 
to the commodities or services. Alternatively, the infor- 
mation pertinent to the prices of the commodities or 
services or the information pertinent to the selling store 
can be confirmed on the picture. 
2 ° [0595] Meanwhile, the number of times of access 
events, displayed in the scene-based access informa- 
tion display column 93B or in the access information dis- 
play column on the commodity or service basis 93D of 
Fig.47, may be the number of times the information is 
25 acquired by the audience first selecting the commodities 
or services and subsequently acting on the commodity 
information site jump button 93G, or the sum total of the 
number of times until the purchase of the commodities 
or services. 

30 [0596] It may be contemplated that the information 
pertinent to the number of times of access events may 
be obtained not only from the information furnishing 
server connected to the premises of the audience over 
a communication network, but also from the latest infor- 
ms mation directly broadcasted or distributed from the 
broadcasting or distributing station or acquired from a 
recording medium which has recorded the information 
at a certain time point. 

[0597] Examples of display pictures, demonstrated 
^ on actuating the meta-cart button jump button 93E, me- 
ta-mall jump button 93F and the commodity information 
site jump button 93G, will be explained later in detail. 
[0598] In the case of Fig.48, the information on the 
uniforms or shoes of a player appearing in an imaged 
« object in a sports program, as well as the institution in- 
formation concerning an arena or gymnasium, is obtain- 
able, in the case of Fig.49, the information concerning 
the performer of the musical sound used in the picture 
program, the information on BGM, or the information 
so concerning the editing technique or the special effect 
technique used in a picture program, is also obtainable. 
[0599] Reference is now made to Fig.50, which differs 
from Figs.47 to 49 in that an index picture 93J is included 
among the objects for display. If a meta-pocket is dis- 
ss played along with the real broadcast, the content of the 
meta-pocket can be confirmed in the real broadcast, 
however, the picture of the real broadcast is usually 
changed. Thus, commodity items can be deliberately 
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selected by having the index picture 93J displayed, even 
when the meta-pocket is displayed along with the real 
broadcast. 

[0600] If the meta-pocket content is confirmed later, 
but the index picture 93J for a scene in question is dis- 5 
played on the same picture along with the information 
pertinent to commodities or services, the necessary in- 
formation can be accessed without the audience having 
to resort to his or her recollections, with the conse- 
quence that the meta-pocket confirmation may be made 
a separate event from the viewing of the real broadcast. 
This testifies to the increased practical value of the me- 
ta-pocket. If the viewer is usually busy, he or she may 
transfer the meta-pocket equivalent portion of the pro- 
gram ancillary data (metadata) of the picture program 
recorded on the household equipment to a portable ter- 
minal to confirm the content outdoors or to offer to pur- 
chase the commodities or services he or she is interest- 
ed in. The index picture 93 1, if displayed, is convenient 
in such case because the scene content can then be 
comprehended reliably. 

(2) M eta-mall display example 

[0601] Referring to Figs.51 and 52, examples of dis- 
play pictures for displaying a list of the information on 
the commodities or services for plural scenes on the 
same picture, that is display examples of meta-malls, so 
termed in the present specification, are explained. The 
meta-mall in question may not only be received or re- 
produced as program ancillary data, but also may be 
created in the signal processing in the processing de- 
vice for program ancillary data. 

[0602] First, the layout structure of Fig. 51 is ex- 
plained. A display picture 95 includes a meta-mall dis- 
crimination number display column 95A, as meta-mall 
management information, a meta-pocket discrimination 
number display column 95B, as management informa- 
tion for meta-pockets, envisaged by the meta-mall to be 
displayed in a list, an index picture 95C, representing 
the scene content of each meta-pocket, a display col- 
umn 95D for the information on commodities or servic- 
es, a meta-cart jump button 95E, embedding the link in- 
formation to the meta-cart, a commodity information site 
jump button 95F, embedding the link information to the 
detailed information pertinent to the commodities or 
services as selected on the picture, a purchase button 
95G used for inputting the will to purchase, a scroll but- 
ton 95H, a purchase will cancel button 951 and a meta- 
mall display end button 95J. 

[0603] Since plural scene information are displayed, 
as shown in Fig.51 , the display area for one scene, that 
is meta-pocket, is limited, as a result of which the 
number of items displayed per scene is also limited. Al- 
though a column for illustration of the commodities is 
provided in the display example of Fig.51 as display con- 
tent of the information pertinent to commodities or serv- 
ices, the column for illustration of the commodities may 



also be omitted, as in the aforementioned display exam- 
ples for meta-pockets. Conversely, the column for ex- 
planation of the commodities may also be provided in 
the display picture examples of the meta-pockets as dis- 
play content of the information pertinent to commodities 
or services. 

[0604] In the case of Fig.51, an index picture 95C is 
demonstrated in association with each meta-pocket. 
This facilitates scene confirmation by the audience. The 
index picture may not be provided, in which case a target 
scene must be identified based solely on the text infor- 
mation. 

[0605] One or plural meta-malls 95 may exist for one 
picture program. In any case, if all scenes making up 
the meta-mall cannot be demonstrated at a time, the 
scroll button 95H on the picture may be used to confirm 
the target scene. 

[0606] Fig. 52 shows a second display example of a 
meta-mall 95. Fig.52 differs from Fig.51 in that the meta- 
mall display end button 95J is displayed in each scene, 
and in that a commodity- or service-based access infor- 
mation display column 95K is displayed for each of the 
commodities or services. 

[0607] The information on the sub-total of the num- 
bers of times of access events, thus displayed from 
scene to scene, can be used as an index as to which is 
the scene where favorite commodities exist and also as 
an index as to which are favorite commodities or serv- 
ices in each scene, as shown in Fig.52. 

(3) Meta-cart display examples 

[0608] Fig.53 shows a display example of a meta-cart 
picture displayed on actuation of a meta-cart jump but- 
ton on a meta-pocket picture or on a meta-mall picture. 
The meta-cart is a picture used for the viewer to re-con- 
firm the content of the commodities or services to be 
purchased before the viewer fin ally decides on the com- 
mercial transaction on the commodities or services he 
or she has expressed the will to purchase. 
[0609] Such a display picture 97 includes a meta- 
pocket discrimination number display column 97A, for 
specifying a scene to which belongs the commodity de- 
sired to be purchased, a scene-based access informa- 
tion display column 97B, demonstrating the number of 
times of access events to a corresponding particular 
scene, an index picture 97C representing the content of 
each scene, an commodity- or service-based access in- 
formation display column 97D, a commodity or service 
information display column 97E, a meta-mall jump but- 
ton 97F embedding the link information to the meta-mall, 
a purchase button 97G used for inputting the will to pur- 
chase, a commodity information site jump button 97H, 
embedding the link information to the detailed informa- 
tion pertinent to commodities or services selected on the 
picture, a scroll button 97I and a meta-cart display end 
button 97J. 

[061 0] Since the index picture 97C is displayed in this 
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manner in the meta-cart as a basket picture, the content 
of the commodities or services to be purchased can be 
confirmed as a picture. Thus, in case the content of the 
commodities or services is desired to be confirmed, it is 
unnecessary to revert to the picture in which the pur- s 
chase has been entered, thus saving laborious inputting 
operations. Moreover, since this means saving in time 
needed until final decision of the wild to purchase, the 
commercial transaction may be concluded in a shorter 
time, which is a merit for the business operator acting 10 
as an intermediary for commercial transactions. 
[061 1 ] Since the scene-based or commodity- or serv- 
ice-based access information is displayed, it is possible 
to check for popularity of the commodities or services 
among other viewers. So, when the viewer desirous to 15 
confirm the popularity for given commodities or servic- 
es, he or she is not in need to revert to the picture in 
which he or she offered a purchase, thus saving labori- 
ous inputting operations. Meanwhile, there is no neces- 
sity for displaying the scene-based access information. 20 
[0612] Although only the meta-mall jump button 97F 
is displayed, the meta-pocket jump button embedding 
the link information to the meta-pocket may afso be dis- 
played, while both the meta-mall jump button 97 F and 
the meta-pocket jump button may be displayed. 25 

(4) commodity information site display example 

[0613] Fig.54 shows a display picture example of a 
commodity information site. This display picture is used 30 
for allowing to confirm the detailed information in case 
the information displayed in the meta-pocket or in the 
meta-mall falls short. The information in the scene in 
question may be stored in a server for which each busi- 
ness operator furnishing the commodities or services 35 
has a home page, or in a server of the brokerage busi- 
ness operator acting as an intermediary for the commer- 
cial transactions between the audience and the busi- 
ness operator. 

[061 4] A display picture 98 includes a scene informa- 40 
tion display column 98A, pertinent to the commodities 
or services in question (as link source), a commodity- 
or service-based access information display column 
98B, displaying the access state for each commodity, a 
picture display column 98C, demonstrating an image 45 
picture of each commodity and an index picture of a cor- 
responding scene, a commodity/service display column 
98D demonstrating the content of the commodities or 
services, a meta-pocket jump button 98E, embedding 
the link information to the meta-pocket, a meta-mall so 
jump button 98F, embedding the link information to the 
meta-mall, a meta-cart jump button 98G, embedding the 
link information to the meta-cart, a purchase button 98H 
used for inputting the will to purchase, a detailed infor- 
mation display button 98! for displaying the further infor- 55 
mation, if any, a sponsor business operator site jump 
button 98J embedding the link information to a sponsor 
business operator site, a scroll button 98K, and an end 
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button 98L used for commanding the end of display of 
the commodity information site. 
[0615] In the commodity- or service-based access in- 
formation display column, there may be displayed the 
cumulative access information managed by a business 
operator furnishing the commodities or services, or the 
information owned by a brokerage business operator. 
[061 6] Although Fig.54 shows only one picture show- 
ing the content of commodities or services in the picture 
display column 98C, plural pictures may, of course, be 
displayed. 

(5) Other display examples 

[0617] Figs.55, 56 show other examples of display 
pictures. Fig.55 shows a picture example for facilitating 
the access of the audience to a meta-mall picture. A dis- 
play picture 99 includes a real-time meta-mall column 
99A, for displaying the meta-mall pertinent to a televi- 
sion program, being broadcasted, and an already aired 
meta-mall column 99B, for retrieving and displaying the 
already aired meta-mall picture. 
[0618] The information displayed in the real-time me- 
ta-mall column 99A may be inserted at a preset interval 
in the received data, or distributed from the terminal of 
the brokerage business operator on request from the 
audience. On the other hand, the information displayed 
in the already aired meta-mall column 99 B may be the 
information received by the audience in the past and 
stored on a recording medium, or the information dis- 
tributed from the terminal of the brokerage business op- 
erator on request from the audience. In the case of Fig. 
55, each of the airing date, time zone and the channel 
is given as an item of selection, and the necessary in- 
formation is read out and displayed at a time point of 
finally determined selection. 

[0619] A display picture 1 00, shown in Fig.56, repre- 
sents a display example when the meta-pocket 1 00A, a 
meta-mall 100B and a real broadcast picture 100C are 
arrayed on one display picture. Since the above infor- 
mation can be confirmed simultaneously on the display 
picture, the viewer is able to acquire or purchase the 
detailed information concerning his or her favorite com- 
modities or services as he or she enjoys the real broad- 
cast. Meanwhile, there are displayed, in the real broad- 
cast picture 1 00C, an index picture of a scene corre- 
sponding to a meta-pocket, an image picture provided 
for commodities or services selected on the meta-pock- 
et or on the meta-mall, and so forth, in addition to the 
pictures of the real broadcast. 

(c) Processing operations 

[0620] Fig. 57 shows an example of a processing op- 
eration at the time of confirming the content of the pro- 
gram ancillary data in the processing device 85 for pro- 
gram ancillary data, as an audience side terminal, spe- 
cifically, at the time of demonstration of the meta-pocket 
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or the m eta-mall. 

[0621] On accepting an operating input by the audi- 
ence, the controller 85F verifies whether or not the com- 
mand contents are requesting a list display of the infor- 
mation on the plural pictures, that is requesting display 
of a meta-mall (step S50). If the negative result is ob- 
tained in the check, that is if it is verified that the display 
of a meta-pocket is requested, the controller 85 F exe- 
cutes the processing of specifying a scene to be dis- 
played (step S51). For example, the controller 85 F cap- 
tures the scene specifying information accorded to the 
picture program being displayed at the time point of the 
operation (UMID or time code information) to specify a 
scene to be displayed. 

[0622] When the scene to be displayed is final, the 
controller 85 F at step S52 reads out program ancillary 
data which has the same scene specifying information 
as the specified scene specifying information (UMID or 
time code information) and which is relevant to the com- 
modities or services, orthe program ancillary data which 
has the scene specifying information contained in the 
specified scene specifying information and which is rel- 
evant to the commodities or services (step S52). 
[0623] The controller 85F then reads out the informa- 
tion on the number of times of access events pertinent 
to the specified scene (step S53). The so read-out in- 
formation is used not only for display but also for various 
signal processing operations. For example, the read-out 
information is used for calculating the number of times 
of scene-based access events to display the calculated 
results. For the calculations, the access information an- 
nexed to each of the commodities or services is used. 
The information is also used for displaying the informa- 
tion on the number of times of access events on a meta- 
cart or the like picture. 

[0624] Moreover, the controller 85 F executes the 
processing of creating an index picture for confirming a 
specified picture or of reading out the index picture from 
the program ancillary data (step S54). If the audience 
has the rights for selection of the index picture or if no 
index picture is contained in the program ancillary data, 
as received or reproduced, the processing of creating 
the index picture is executed. 

[0625] Following these operations, the controller 85F 
arrays the read-out information based on the layout in- 
formation for displaying the information for one scene 
(layout information for meta-pocket) to display the so ar- 
rayed information on the display picture (step S55). 
Meanwhile, in case the index picture or the number of 
times of access events is already contained in the pro- 
gram ancillary data as received or reproduced, and can 
directly be displayed, the display picture is first specified 
and subsequently the corresponding data is displayed. 
[0626] In case the positive results are obtained in the 
verification operation of step S50, that is in case it is 
found that the display of a meta-mall is requested, the 
controller 85F executes the processing of specifying 
plural scenes to be displayed (step S56). For example, 



plural past pictures, beginning from the time point of ac- 
tuation, are decided to be displayed, orthe scene infor- 
mation specified by the audience for display is read out 
and displayed. If the information of the scene to be dis- 
5 played is contained in the program ancillary data, the 
information is used. 

[0627] When the scene to be displayed is final, the 
controller 85F at step S57 reads out program ancillary 
data bearing the same scene specifying information as 
10 the specified scene specifying information (UMID or 
time code information) and which is relevant to the com- 
modities or services, orthe program ancillary data hav- 
ing the scene specifying information contained in the 
specified scene specifying information and which is rel- 
evant to the commodities or services. It is noted that the 
information on the commodities or services is read out 
for each of the plural scenes. 

[0628] The controller 58 F then reads out the relevant 
information on the number of times of access events for 
each of the specified scenes (step S58). The so read- 
out information is used not only for display but also for 
variegated signal processing operations. For example, 
the information is used for calculating the scene-based 
number of times of access events within a terminal for 
display. For the calculations, the access information an- 
nexed to each of the commodities or services is used. 
The information is also used for displaying the informa- 
tion on the number of times of access events on a meta- 
cart or the like picture. 

[0629] The controller 85 F executes the processing of 
creating an index picture within a terminal for reading 
out the index picture from the program ancillary data 
(step S59) for confirming a specified scene. If the audi- 
ence has the rights for selection of the index picture or 
if no index picture is contained in the program ancillary 
data as received or reproduced, the processing of cre- 
ating the index picture is executed. 
[0630] Following these operations, the controller 85F 
arrays the read-out information based on the layout in- 
formation for displaying the information for plural scenes 
(layout information for meta-mall) to display the so ar- 
rayed information on the display picture (step S60). 
Meanwhile, in case the index picture or the number of 
times of access events is already contained in the pro- 
gram ancillary data as received or reproduced, and can 
directly be displayed, the display picture is first specified 
and subsequently the corresponding data is displayed. 

(D-2) Server side terminal 

(a) Terminal structure 

[0631 ] The structure of a server device, provided be- 
tween the audience and the picture producer or between 
the audience and the sponsor business operator and 
which furnishes the information to the audience or pros- 
ecutes brokerage of the commercial transactions to 
form a brokerage business operator side system, is 
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hereinafter explained. 

[0632] Rgs.58 and 59 show two e mbodiments of the 
server device, using the same refer ence numerals for 
common parts or components. Meanwhile, the differ- 
ence between Figs.58 and 59 is whether a commodity/ 
server database 105E is provided in the server device 
or on the network. 

[0633] A server device 105 includes an information 
furnishing unit 105A, a layout information holding unit 
105B, a controller 105C, an access inf ormation acqui- 
sition unit 105D, a commodity/service database 105E, 
a communication function unit 105F, a list file creation 
unit 1 05G, an index picture creation unit 1 05H, an index 
picture recording unit 1051, a access event number of 
times calculating unit 105J and a bus 105K. 
[0634] Although display means or an operating input 
unit is not shown in Figs.58 or 59, these aire provided in 
actuality for information inputting or management. Al- 
though Fig.58 shows that the totality of tine functional 
units are provided in the main body unit of the server 
device, part of the functional units may be provided ex- 
ternally. The connection configuration may be wired or 
wireless. 

[0635] The information furnishing unit 1 05A is a func- 
tion used when a data file equivalent to the aforemen- 
tioned meta-mall is to be furnished from the side server 
device. The information furnishing unit 105A is means 
responsive to the audience' request to read out from the 
commodity/service database 105E the file data perti- 
nent to a corresponding scene of a picture program 

t specified by the audience so as to furnish the so read- 
out file data. The file data is an information file provided 
for each picture program for presenting the information 
pertinent to plural scenes to the audience in a list form. 
Meanwhile, the file data may be furnished from the pic- 
ture producer for storage in the database, or may be cre- 
ated and stored in the list file creation unit 105G. 
[0636] The layout information holding unit 105B is a 
recording or storage medium used for creating a data 
file corresponding to the aforementioned meta-pocket 
or meta-mall on the side server device for recording the 
layout information furnished to the audience. The layout 
information so held may be used only for the meta-pock- 
et or only for the meta-mall. The medium may be used 
for recording or storing other information than the layout 
information, without being dedicated to the holding of 

the layout information. 

[0637] These recording mediums include a CD-ROM, 
a magnetic recording type medium, such as tape, floppy 
disc, hard disc or a magnetic card, an optical recording 
medium, such as write-once CD, DVD or optical card, 
magneto-optical recording type medium, such as DVD, 
MD or MO, and a semiconductor memory, such as non- 
volatile memory or RAM. 

[0638] The controller 1 05C is means for controlling 
various parts or components making up the server de- 
vice, and is routinely configured as a micro-computer. 
Usually, the function of the controller 1 05C is prescribed 



by an operation system program and an application pro- 
gram. 

[0639] The access information acquisition unit 105D 
is means for inputting the latest values of the access 
s information pertinent to the commodities or services as 
requested by the audience from the commodity/service 
database 105E. The information as acquired is sent 
through the communication function unit 105Fto the re- 
questing audience. The information furnished may be 
10 the accessing information by itself or the accessing in- 
formation plus the detailed information pertinent to the 
commodities or services as requested by the audience. 
When the picture displayed on the audience' picture is 
to be created on the side server device, the access in- 
15 formation is pasted at a preset position which is in keep- 
ing with the layout information. 

[0640] The commodity/service database 105E is a 
database in which there is recorded the detailed infor- 
mation pertinent to the commodities or services appear- 
20 ing on an imaged object of a picture program. The com- 
modity/service data base may be used in common with 
the database used in the aforementioned database 
used in turn in the aforementioned sponsorship trans- 
action support system. The information in question may 
25 be acquired from the sponsorship transaction support 
system or from the business operator who furnished the 
commodities or services to the picture program. Al- 
though the information may, of course, be input by a su- 
pervisor of the server device, the pre-existing informa- 
30 tion may be used to enable efficient database manage- 
ment. 

[0641] In the commodity/service database 105E, 
there may be recorded the program ancillary data 
(equivalent to a meta-pocket) associated with each 
35 scene of the picture program. Alternatively, the data may 
be held in a medium distinct from that of the commodity/ 
service database 1 05E. 

[0642] The communication function unit 105F is 
means for communication with outside over a network. 
40 As the communication function unit 105F, such a com- 
munication equipment which is in keeping with the net- 
work is used. The network may be a telephone network, 
an IP network, a dedicated line network or an ATM net- 
work. The communication system may be network con- 
4 5 nectionsystemorapacketconnectionsystem. Thenet- 
work may be of an analog or a digital configuration. The 
communication function unit 105F may be externally 
mounted, instead of being enclosed, as shown in Fig. 
58. The access information, meta-mall picture or the in- 
50 dex information is transmitted through this communica- 
tion function unit 105F. 

[0643] The list file creation unit 1 05G is means for cre- 
ating a datafile equivalent to the meta-mall in the present 
specification. The picture forming the meta-mall may be 
55 determined by an operator of the server device, or spec- 
ified from the audience side. There is also such a meth- 
od in which, when the information specifying the time 
point of request of display of the list file from the audi- 
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ence side, that is the information specifying the picture 
program and the scene, is obtained, the scene of use is 
determined based on the rule of preparation held by the 
list file creating unit 10. There is also a method in which 
plural forward pictures are specified from the notifying 
scene as being a meta-mall forming scene, or a method 
in which plural forward and backward scenes are spec- 
ified as being the meta-mall forming scene. 
[0644] The list file creation unit 1 05G arrays the meta- 
pocket data of a specified scene in accordance with the 
layout information for the meta-mall, in order to com- 
plete picture data which is used as a list file. Alternative- 
ly, the list file creation unit 1 05G groups meta-ppcket da- 
ta (file) as found on retrieving meta-pocket data and the 
layout information for a meta-pocket of the specified 
scene into one file to form a list file. 
[0645] The index picture creation unit 105H is means 
used for the brokerage server device to create a meta- 
pocket or a meta-mall provide with an index picture. 
However, solely the index picture may be used by itself 
for furnishing to the audience. The picture or the picture 
material used as an index picture is routinely supplied 
from the picture producer. In case of embedding a mov- 
ing picture reproducing function or the audio reproduc- 
ing function in the index picture, the link information for 
a moving picture file or an audio file may also be pre- 
pared. 

[0646] The index picture recording unit 1051 is a re- 
cording or storage medium for recording an index- pic- 
ture created by the index picture creation unit 105H or 
an index picture furnished from the picture producer. 
However, the medium may be used for recording or stor- 
ing other information than the layout information, without 
being dedicated to the holding of the index picture. 
These recording mediums include a CD-ROM, a mag- 
netic recording type medium, such as tape, floppy disc, 
hard disc or a magnetic card, an optical recording me- 
dium, such as write-once CD, DVD or optical card, mag- 
neto-optical recording type medium, such as DVD, MD 
or MO, and a semiconductor memory, such as non-vol- 
atile memory or RAM. 

[0647] The access event number of times calculating 
unit 105J is means for monitoring the readout request 
(access) of the information on the commodities or serv- 
ices to the commodity/service database 1 05E by the au- 
dience or the state of conclusion of agreements to up- 
date the number of access events held by the commod- 
ity/service database 1 05E or to calculate the number of 
times of scene-based access events. For example, in 
case the detailed information concerning the commod- 
ities or services is furnished over the Internet, the unit 
is implemented by an application such as CGI (common 
gateway interface). The operational sequence in case 
of utilizing the CGI is carried out as follows: 
[0648] For example, if it is verified that a viewer has 
requested furnishment of the detailed information on the 
commodities or services, the access event number of 
times calculating unit 105J is started (access counter 



CG1 is started), based on the statement of the access 
counter contained in the furnished information, before 
execution of transmission of the relevant information to 
the audience. So, the access counter CG1 , thus started, 
s increments the count value of the information furnished, 
here the detailed information on commodities or servic- 
es. It is the access event number of times calculating 
unit 105J that executes this processing. 
[0649] The number of times of access operations, 
10 thus calculated, is used not only as a list file or as the 
detailed information on the commodities or services, but 
may by itself be furnished to the audience. If the audi- 
ence reproduces and uses the program ancillary data 
recorded on a recording medium, it is useful to sepa- 
rately calculate the number of access events for furnish- 
ment to the audience. That is, the viewer is able to use 
the latest information as a reference for purchase. 
[0650] The bus 105K is a data input/output bus, such 
as, for example, an ISA-, an EISA- or a PCI bus. 
[0651] The server device 105, shown in Fig. 58, may 
be formed by computer terminals, such as notebook or 
desktop personal computers, PDA, portable telephone 
sets, game machines, set top boxes, PC cards, domes- 
tic electrical utensils, VTR, television receivers, car nav- 
igation systems, or wrist watches having the information 
processing functions, in addition to the so-called dedi- 
cated terminal. 

[0652] The server device 1 05, shown in Fig.59, is ex- 
plained. In the case of the server device 105, the com- 
modity/service database 105E is connected over a net- 
work to the server device 1 05. This configuration is used 
in general for a case wherein the business operator ex- 
ecuting the services of furnishing the information to the 
audience or the services of acting as an intermediary for 
commercial transactions differs from the business oper- 
ator managing the commodity/service database 105E. 
However, the connection configuration shown in Fig.59 
may be used even if the two business operators are the 
same. 

(D-3) processing operations between terminals 

[0653] Figs. 60 and 61 show how processing pro- 
ceeds between the audience side primary or secondary 
terminal and the server device when the meta-pocket or 
meta-mall pictures are displayed. 
[0654] Fig. 60 shows meta-pocket display and an ex- 
emplary processing sequence until the end of commer- 
cial transaction. First, the audience perceives the infor- 
mation as to the presence of the program ancillary data 
concerning commodities or services appearing in the 
picture program being viewed and commands a meta- 
pocket picture to be displayed through e.g., a remote 
controller (step S65). The display command is notified 
to an audience side terminal device (program ancillary 
data processing device). 

[0655] On receipt of the display command, the termi- 
nal device detects an identification number (ID) of the 
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meta-pocket picture to be displayed, based on the 
scene information of the picture program (step S66). On 
detection of the meta-pocket ID, the terminal device 
reads out relevant data, based on the detected meta- 
pocket ID (step S67). In the absence of the correspond- 
ing relevant data, the terminal device requests the ex- 
ternal database (which is not necessarily a shopping 
brokerage server) to download the information. At any 
rate, when the data relevant to the meta-pocket ID is 
obtained, a meta-pocket picture, correlated to the scene 
commanded by the audience, appears on a display pic- 
ture of the terminal device (step S68). 
[0656] In the meta-pocket picture, the information 
concerning plurai imaged objects, presented on a scene 
of a picture program, is displayed on one picture. On the 
meta-pocket picture, an index picture indicating the con- 
tent of a scene, the accessing information concerning 
each imaged object and the scene-based access infor- 
mation, are displayed. 

[0657] If now the audience specifies some commodi- 
ties or services on the picture and commands further 
detailed information to be displayed (step S69), the 
command is transmitted to a shopping brokerage server 
through the terminal device overthe network. On receipt 
of the request, the shopping brokerage server updates 
the value of the access counter, representing the level 
of the interest the audience has in the relevant informa- 
tion (step S70), and transmits the requested commodity 
data to the terminal device who is the source of request 
(step S71 ). The terminal device demonstrates the infor- 
mation of the notification on the picture (step S72). 
[0658] When subsequently the viewer on reviewing 
the display content is inclined to purchase and depress- 
es the purchase button (step S75), the actuation of the 
purchase button is notified to the terminal device so that 
the information on the commodities or services being 
purchased is registered in the meta-cart (step S74). 
When the meta-cart information is to be imparted to the 
shopping brokerage server, the actuation and the infor- 
mation on the object of purchase are notified to the 
shopping brokerage server. At this stage, the commer- 
cial transaction between the viewer and the shopping 
brokerage business operator has not as yet been finally 
determined. 

[0659] If then the viewer requests confirming the in- 
formation on the commodities or services, registered in 
the meta-cart (step S75), the command operation is no- 
tified to the terminal device, so that the information reg- 
istered in the meta-cart is displayed on the picture (step 
S76). Meanwhile, in case the meta-cart information is 
held on the shopping brokerage server, the recorded in- 
formation is notified from the shopping brokerage server 
and displayed. At this time, the index picture represent- 
ing the scene of the picture program, which gave a clue 
to the purchase of the commodities or services regis- 
tered on the meta-cart picture, the commodity- or serv- 
ice-based access information and the scene-based ac- 
cess information are displayed. 



[0660] If then the viewer confirms the content of the 
commodities or services registered in the meta-cart and 
re-confirms the will to purchase, that is if the viewer acts 
on the purchase button (step S77), the information on 
s the operation is notified through the terminal device to 
the shopping brokerage server. The shopping broker- 
age server, who has received the notice, executes the 
processing of ultimately accepting the information on 
the purchaser and the information on the commodities 
10 or services being purchased (Step S78). 

[0661] The registered information is subsequently 
used as a reference material for subsequent sponsor- 
ship transaction or as automatic setting parameters for 
the distribution area. On registration of the acceptance 
15 of the offer for purchase (consent to purchase) by the 
shopping brokerage server, the notice of the end of the 
registration of the acceptance is returned to the terminal 
device in question. This information is displayed on the 
display picture of the terminal device (step S79). 
20 [0662] Fig. 61 shows the display of a meta-pocket and 
an illustrative processing sequence until the end of the 
commercial transaction. Meanwhile, Fig.61A shows a 
case where the display content of the meta-mall are al- 
ready determined at the time of the operation and Fig. 
25 61 B shows a case where the display content of the me- 
ta-pocket picture is determined subject to command. 
[0663] In Fig.61A, the audience perceives the infor- 
mation as to the presence of the program ancillary data 
concerning commodities or services appearing in the 
30 picture program being viewed and commands a meta- 
mall picture to be displayed through e.g., a remote con- 
troller (step S80). The display command is notified to a 
audience side terminal device (program ancillary data 
processing device). 
35 [0664] On receipt of the display command, the termi- 
nal device detects an identification number (ID) of the 
meta-mall picture to be displayed, based on the scene 
information of the picture program (step S81). On de- 
tection of the meta-mall ID, the terminal device reads 
40 out relevant data, based on the detected meta-mall ID 
(step S82). In the absence of the corresponding relevant 
data, the terminal device requests the external database 
(which is not necessarily a shopping brokerage server) 
to download the information. At any rate, when the data 
45 relevantto the meta-mall ID is obtained, a meta-mall pic- 
ture, correlated to the scene commanded by the audi- 
ence, appears on a display picture of the terminal device 
(step S83). 

[0665] In the meta-mall picture, the information con- 
50 ceming plural scenes is displayed on one picture such 
as to permit confirmation. Of course, the information on 
plural commodities or services appearing in one scene 
is displayed in the information on the commodities or 
services corresponding to each scene. In each scene, 
55 the index information representing the content, the ac- 
cess information for each imaged object and the scene- 
based access information are displayed. The ensuing 
operation is similar to that explained in connection with 
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Fig. 60 and hence is not explained specifically. 
[0666] In the case of Fig. 61 B, the viewer perceives 
the information as to the presence of the program ancil- 
lary data concerning commodities or services appearing 
in the picture program being viewed and commands a 
meta-mall picture to be displayed through e.g., a remote 
controller (step S80). The display command is notified 
to a audience side terminal device (program ancillary 
data processing device). 

[0667] On receipt of the notice on the display com- 
mand, the terminal device specifies a scene position at 
the time point of the command input and, based on the 
so specified scene position as a reference, reads out 
the range information of the scene forming the meta- 
mall picture (step S84). In the case of live broadcast, a 
preset number of pictures present on the time axis pre- 
vious to the time pint when the command for displaying 
the meta-mall picture was made are set as the scene 
range forming the meta-mall picture. In the case of the 
replay broadcast, a preset number of scenes present 
ahead and at back of a time point of the display com- 
mand for the meta-mall picture as the center are set as 
the scene range forming the meta-mall picture. Mean- 
while, if the viewer has previously entered the scenes, 
desired to be displayed as the meta-mall picture, the 
scenes forming the meta-mall picture are set from 
among the previously entered scenes. 
[0668] When the scenes forming the meta-mall pic- 
ture have been specified, the terminal device, for exam- 
ple, detects an identification number (ID) of the meta- 
pocket pictures corresponding to the scenes (step S85). 
On detection of the m eta-pocket ID, the terminal device 
reads out the relevant data based on the m eta-pocket 
ID (step S86). In the absence of the corresponding rel- 
evant data, the terminal device requests the external da- 
tabase (which is not necessarily a shopping brokerage 
server) to download the information. At any rate, when 
the data relevant to the meta-mall ID is obtained, the 
terminal device creates a meta-mall picture, based on 
the relevant data (step S87). 

[0669] On completion of creation of the meta-mall pic- 
ture, the terminal device displays the so-created meta- 
mall picture on the display picture (step S88). The en- 
suing operations are similar to those of Fig. 60 and 
hence are not explained specifically. The layout or the 
display content of the meta-mall picture is similar to 
those shown in Fig.61 A. 

[0670] The processing sequence of Figs. 60 and 61 
are given only by way of illustration, such that it may be 
possible to perform the processing operations conform- 
ing to the service furnishing configurations. 

(D-4) summary 

[0671] By constructing the digital picture program re- 
producing system, as described above, it is possible to 
construct an equipment structure and a man -machine 
interface which are convenient to use for the audience. 



In particular, since the audience is able to acquire the 
information pertinent to plural commodities or services, 
appearing as imaged objects in the respective scenes, 
on a program ancillary data picture associated with one 
5 scene (meta-pocket picture or meta-mall picture) , the to- 
tality of the picture programs other than the television 
shopping programs can be used as a platform for the 
commodity/service information. 

[0672] Moreover, since not only is the information per- 
10 tinent to a scene displayed as program ancillary data on 
a picture, but also the information on plural scenes can 
be confirmed on the same picture (meta-mall picture), 
the target scene can be accessed with a lesser amount 
of operations. This means that a picture may be provid- 
es ed which is convenient to use for a broad range of age 
groups. 

[0673] Since not only is the information on the com- 
modities orservices appearing in each scene furnished, 
but also the access states from the audience to these 
20 commodities or services are furnished as count values, 
the audience can be afforded with the information ben- 
eficial in purchasing the commodities orservices. Mean- 
while, the number of times of access events, collected 
on the scene basis, is the information beneficial in gross- 
es |y selecting the scene. 

[0674] By causing the index picture of a correspond- 
ing scene to be displayed in displaying the information 
on the commodities or services appearing in each 
scene, the viewing of the picture program can be isolat- 
30 ed from the time point of recognition of the program an- 
cillary data, thus improving the convenience in use for 
the audience. In particular, for worker's family units and 
youngsters, for whom the living time in households is 
limited, such use modes of transferring the program an- 
as ciliary data, acquired in the primary terminal, to the sec- 
ondary terminal for confirmation outdoors may be wel- 
comed, in which case the presence of the index picture 
is helpful for the viewer to arrive at the target scene more 
promptly, while the content of the commodities or serv- 
40 ices targeted may be confirmed visually. The result is 
that the number of cases reaching the stage of commer- 
cial transactions is appreciably larger than if the scheme 
of the present invention is not used. 



[0675] With the program ancillary data processing de- 
vice, according to the present invention, in which the de- 
gree of interest the audience at large feels in the com- 

so modities or services appearing in a picture program can 
be checked on the scene basis, it can be confirmed 
readily whether or not commodities or services that are 
in vogue are contained in a given scene. 
[0676] With the program ancillary data processing de- 

55 vice, according to the present invention, in which the de- 
gree of interest the viewers at large feels in the com- 
modities orservices appearing in a picture program can 
be checked from one commodity or service appearing 
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m a given scene to another, It can be readily checked 
whether or not specified commodities or services are al- 
so accepted by other viewers. 

[0677] With the program ancillary data processing de- 
vice, according to the present invention, in which the tar- 
get information relevant to the number of times of ac- 
cess events is obtained from the program ancillary data 
received or reproduced with the picture program, any 
device not carrying communication means may conven- 
iently be used. With the program ancillary data process- 
ing device, according to the present invention, in which 
the information relevant to the number of times of ac- 
cess events can be acquired over a communication net- 
work independently of picture program data, the latest 
information on the number of times of access events can 
be provided for the viewer not only when the contents 
of the program ancillary data are checked as the picture 
program progresses but also when the transiently 
stored picture program data is reproduced or when the 
picture program data distributed as a recording medium 
is reproduced. 

[0678] With the program ancillary data processing de- 
vice, according to the present invention, the viewer is 
able to confirm the contents of the respective scenes 
making up a picture program, in particular the commod- 
ities or services afforded by the program ancillary data, 
as he or she compares the contents to the display in the 
index picture. 

[0679] With the program ancillary data processing de- 
vice, according to the present invention, in which there 
is no necessity of providing the index picture in the de- 
vice itself, the processing resources necessary for exe- 
cuting the processing may be allocated to other 
processings. 

[0680] With the program ancillary data processing de- 
vice, according to the present invention, in which an in- 
dex picture is prepared in the device itself, it is sufficient 
if only an index picture as required is prepared, thus 
evading the wasting of many memory areas by the index 
pictures of the major portion of the program ancillary da- 
ta which do not have to be confirmed. 
[0681] With the program ancillary data processing de- 
vice, according to the present invention, in which not the 
index picture in which commodities or services relevant 
to program ancillary data appear in an averaged out 
fashion, but a picture which permits easy recognition of 
the commodities or services the viewer is desirous to 
view, can be selected as the index picture, the viewer is 
able to select an optimum picture. 
[0682] With the program ancillary data processing de- 
vice, according to the present invention, in which the in- 
formation pertinentto plural scenes can be accessed on 
one picture, it is possible to evade the laborious opera- 
tion of confirming the contents from scene to scene. 
Moreover, since the information on the plural scenes 
can be compared on one picture, the target information 
can be accessed extremely readily. 
[0683] With the program ancillary data processing de- 



vice, according to the present invention, in which the 
viewer is able to confirm the contents of the respective 
scenes making up the picture program, anyone can find 
a scene in which he or she is interested in, from the plu- 
s ral scenes displayed in a list form, in addition, since the 
contents of the commodities or services appearing in a 
specified scene can be checked on the same display 
picture as the viewer compares the contents, the viewer 
feel easy when using the device. 
10 [0684] With the program ancillary data processing de- 
vice, according to the present invention, in which the 
scenes displayed in the list form are only those contig- 
uous to one another, anyone can access the target 
scene extremely readily. 

[0685] With the program ancillary data processing de- 
vice, according to the present invention, only those 
scenes the viewer has selected with the intention to 
make a check may be displayed. 
[0686] With the program ancillary data processing de- 
& vice, according to the present invention, such a picture 
may be provided to the viewer which enables accessing 
to the information concerning plural scenes on one dis- 
play picture, when the information pertinent to commod- 
ities or services appearing in each scene is not con- 
25 tained in the program ancillary data, but data indicating 
whether or not the information pertinent to commodities 
or services appearing in each scene is available through 
an external database is contained in the program ancil- 
lary data, and the data indicates availability of the infor- 
30 mation from the external database. 

[0687] With the program ancillary data processing de- 
vice, according to the present invention, the program an- 
cillary data received or reproduced in another terminal 
device and stored on a recording medium can be sep- 
35 arately taken out and re-used. 

[0688] With the program ancillary data processing de- 
vice, according to the present invention, in which the in- 
formation concerning plural scenes can be checked at 
a time on the display picture, the necessity for process- 
*o ing for displaying the information of other scenes on the 
display picture may be eliminated, even when the target 
information is not contained in the information concern- 
ing a given scene. Thus, it is possible to realize retrieval 
which is more efficient than if each scene is accessed 
45 independently. 

[0689] With the program ancillary data processing de- 
vice, according to the present invention, it is possible to 
check what are the commodities or services scheduled 
to be purchased, on an index picture, when these com- 
50 modities or services are re-checked on the basket pic- 
ture. Consequently, the viewer is freed of the labor of 
exiting the basket picture to check the contents of the 
commodities or services to move to the other picture. In 
particular, when the commodities being checked on the 
55 basket picture affect plural scenes, the operation of re- 
verting to the respective scenes may be eliminated. This 
gives a picture for check that can be easy to operate 
and use for anyone. 
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[0690] With the server device according to the present 
invention, even when the information on the access 
state from the viewer to the detailed information relevant 
to the commodities or services is not included in the pic- 
ture program data received or reproduced by the viewer, 5 
it is possible to render services employing the informa- 
tion. 

[0691 ] With the server device of the present invention, 
the latest value of the access state from the viewer to 
the detailed information pertinent to the commodities or 
services appearing as an imaged object in the picture 
program can be incidentally furnished responsive to a 
request from the viewer 

[0692] With the server device of the present invention, 
even when the information file is not included in the pic- 
ture program data received or reproduced by the viewer, 
the services employing the file may be provided to the 
viewer. 

[0693] With the server of the present invention, it is 
possible to provide a list file in which an index picture is 
added to each scene. Since the index picture is affixed 
to each of plural scenes making up the list file, scene 
retrieval by the viewer may be expectantly facilitated. 
[0694] With the program information display method 
of the present invention, the viewer is able to judge eas- 
ily, without regard to the route of acquisition of the 
number of times of access events, whether or not pop- 
ular commodities or services are contained in the scene 
on the display picture he or she is interested in. The pro- 
gram information display method of the present inven- 
tion is useful in particular in giving a decision, when 
browsing the information on the commodities or servic- 
es appearing in the imaged object in many scenes, as 
to in which scene the information is to be browsed. 
[0695] With the program information display method 
of the present invention, the viewer is able to judge eas- 
ily, without regard to the route of acquisition of the index 
picture, which specified commodities or services ap- 
pearing in a picture program are popular among other 
viewers 

[0696] With the program information display method 
of the present invention, the viewer is able to judge eas- 
ily, without regard to the route of acquisition of the index 
picture, which commodities or services appear in each 
scene. When the information pertinent to commodities 
or services is to be checked, the contents can be 
checked easily because the index picture, usually given 
as a still picture, is displayed. 

[0697] With the program information display method 
of the present invention, in which the information perti- 
nent to plural scenes can be checked at a time on the 
display picture, the operation of causing the information 
of another scene to be displayed on the display picture 
may be dispensed with even when the target information 
is not included in the information pertinent to a given 
scene. Thus, more efficient retrieval can be realized 
than if the respective scenes are accessed independ- 
ently. 



[0698] With the program information display method 
of the present invention, in which the target scene can 
be selected as the contents of the respective scenes are 
checked on the index picture, the operational efficiency 
can be improved. In checking the information concern- 
ing the commodities or services, their contents can be 
visually checked visually on the index picture. 
[0699] With the recording medium of the present in- 
vention, any pre-existing terminal can be used as a pro- 
gram ancillary data processing device, by simply install- 
ing a program recorded on the recording medium in the 
pre-existing terminal, even in the absence of a dedicat- 
ed program ancillary data processing device. 
[0700] With the recording medium of the present in- 
vention, any pre-existing terminal can be used as a pro- 
gram ancillary data processing device, by simply install- 
ing a program recorded on the recording medium in the 
pre-existing terminal, even in the absence of a dedicat- 
ed program ancillary data processing device. 
[0701] With the recording medium of the present in- 
vention, any pre-existing terminal can be used as a pro- 
gram ancillary data processing device, by simply install- 
ing a program recorded on the recording medium in the 
pre-existing terminal, even in the absence of a dedicat- 
ed program ancillary data processing device. 
[0702] With the recording medium of the present in- 
vention, any pre-existing terminal can be used as a pro- 
gram ancillary data processing device, by simply install- 
ing a program recorded on the recording medium in the 
pre-existing terminal, even in the absence of a dedicat- 
ed program ancillary data processing device. 
[0703] With the recording medium of the present in- 
vention, any pre-existing terminal can be used as a pro- 
gram ancillary data processing device, by simply install- 
ing a program recorded on the recording medium in the 
pre-existing terminal, even in the absence of a dedicat- 
ed program ancillary data processing device. 
[0704] With the recording medium of the present in- 
vention, any pre-existing terminal can be used as a pro- 
gram ancillary data processing device, by simply install- 
ing a program recorded on the recording medium in the 
pre-existing terminal, even in the absence of a dedicat- 
ed program ancillary data processing device. 
[0705] With the recording medium of the present in- 
vention, any pre-existing terminal can be used as a pro- 
gram ancillary data processing device, by simply install- 
ing a program recorded on the recording medium in the 
pre-existing terminal, even in the absence of a dedicat- 
ed program ancillary data processing device. 
[0706] With the recording medium of the present in- 
vention, any pre-existing terminal can be used as a pro- 
gram ancillary data processing device, by simply install- 
ing a program recorded on the recording medium in the 
pre-existing terminal, even in the absence of a dedicat- 
ed program ancillary data processing device. 
[0707] With the recording medium of the present in- 
vention, any pre-existing terminal can be used as a pro- 
gram ancillary data processing device, by simply install- 
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mg a program recorded on the recording medium in the 
pre-existing terminal, even in the absence of a dedicat- 
ed program ancillary data processing device. 

Claims 

1. A program ancillary data processing apparatus 
comprising: 

a program ancillary data extraction unit for ex- 
tracting, from picture program data, program 
ancillary data relevant to commodities or serv- 
ices appearing in each scene; and 
a display control unit for displaying, on adisplay 
picture, the information relevant to the com- 
modities or services appearing in each scene, 
and the sum total of the number of times of 
scene-based access events by the audience to 
a database which supervises the detailed infor- 
mation relevant to said commodities or servic- 
es, based on the preset layout information. 

2. A program ancillary data processing apparatus 
comprising: 

a program ancillary data extraction unit for ex- 
tracting, from picture program data, program 
ancillary data relevant to commodities or serv- 
ices appearing in each scene; and 
a display control unit for displaying, on a display 
picture, the information relevant to the com- 
modities or services appearing in each scene, 
andthesum total of the n umbe r of times of com- 
modity-based or service-based access events 
by the audience to a database which supervis- 
es the detailed information relevant to said 
commodities or services, based on the preset 
layout information. 



5. 



3. 



4. 
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The program ancillary data processing apparatus 
according to claim 1 or 2 wherein 

the results of the summation of the number of 
times of scene-based access events, the summa- 
tion of the number of times of commodity-based ac- 45 
cess events and the summation of the service- 
based access events by the audience to the data- 
base, supervising the detailed information on the 
commodities or services, are acquired as program 
ancillary data of the picture program data. so 

The program ancillary data processing apparatus 
according to claim 1 or 2 wherein 

the results of the scene-based summation, 
commodity-based summation or the service-based ss 
summation of the numbers of times of access 
events by the audience to the database, supervis- 
ing the detailed information pertinent to the com- 



modities or services, are acquired, apart from the 
picture program data, from the database which is 
connected to the apparatus over a communication 
network and which supervises the detailed informa- 
tion on the commodities or services. 

A program ancillary data processing apparatus 
comprising: 

a program ancillary data extraction unit for ex- 
tracting, from picture program data, program 
ancillary data relevant to commodities or serv- 
ices appearing in each scene; and 
a display control unit for displaying the informa- 
tion relevant to the commodities or services ap- 
pearing in each scene, on a display picture, 
along with an index picture of a corresponding 
scene, based on the preset layout information. 

The program ancillary data processing apparatus 
according to claim 5 wherein said index picture is 
given as program ancillary data for the picture pro- 
gram data. 

The program ancillary data processing apparatus 
according to claim 5 wherein said index picture is 
created on the viewer side from picture data of pic- 
ture program data. 

The program ancillary data processing apparatus 
according to claim 7 wherein said index picture has 
been produced from a scene optionally selected by 
a viewer . 



35 9. A program ancillary data processing apparatus 
comprising: 



a program ancillary data extraction unit for ex- 
tracting from picture program data program an- 
cillary data relevant to commodities or services 
appearing in each scene; and 
a display control unit for displaying the informa- 
tion relevant to plural scenes, as extracted, 
based on the layout information for list type dis- 
play. 

10. The program ancillary data processing apparatus 
according to claim 9 wherein 

said display control unit has the function of 
displaying the information relevant to commodities 
or services appearing in each scene, along with an 
index picture of a corresponding scene, on a display 
picture. 

11. The program ancillary data processing apparatus 
according to claim 9 wherein scenes of the picture 
program to be processed are extracted in groups 
composed of a preset number of scenes, in the or- 
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der in which the scenes are displayed, based on the 
layout information, for presentation as plural scenes 
for the audience in a list form. 

12. The program ancillary data processing apparatus 
according to claim 9 wherein scenes of the picture 
program to be processed are optionally command- 
ed for selection by a viewer and extracted in groups 
composed of a preset number of scenes, based on 
the layout information, for presentation as plural 
scenes for a viewer in a list form. 

13. A program ancillary data processing apparatus 
comprising; 

an extraneous data presence/absence detec- 
tion unit for extracting, from picture program da- 
ta, data indicating whether or not the informa- 
tion relevant to commodities or services ap- 
pearing in each scene is available through an 
external database, and for detecting the pres- 
ence/absence of the extraneous data based on 
the so extracted data; 

a downloading unit for downloading, when an 
access request to extraneous data is made by 
an operating input by a viewer, a list file provid- 
ed from the outset for presenting the informa- 
tion relevant to plural scenes to in a list form, 
from an external database, said list file being 
an information file provided from the outset for 
a relevant picture program; and 
a display control unit for opening the download- 
ed list file for displaying the information relevant 
to the plural scenes involved in said list file on 
a display picture in a list form, based on the pre- 
set layout information. 

14. A program ancillary data processing apparatus 
comprising: 

a downloading unit for communication with an- 
other terminal device for downloading the infor- 
mation relevant to commodities or services ap- 
pearing in each scene from a recording medium 
supervised by said other terminal device, said 
information being program ancillary data of a 
picture program in question; and 
a display control unit for demonstrating the in- 
formation relevant to commodities or services 
appearing in each scene, on a display picture, 
responsive to an operating input by an operator. 

15. The program ancillary data processing apparatus 
according to claim 14 wherein 

said display control unit has the function of 
presenting the information relevant to plural scenes 
to a viewer in a list form based on the preset layout 
information. 



16. A program ancillary data processing apparatus for 
displaying the information relevant to commodities 
or services appearing in each scene, on a display 
picture, based on program ancillary data extracted 
s from picture program data, said apparatus compris- 
ing: 



a display control unit for displaying an index pic- 
ture of a scene, in an imaged object of which 
fo are included commodities or services selected 

with a will for purchasing, in a correlated fash- 
ion in a basket picture in which the contents of 
said commodities or services selected can be 
confirmed prior to final decision of transaction. 



15 



17. A server apparatus comprising: 



an access information acquisition unit for ac- 
quiring the information pertinent to the access 

20 state from the audience to a database super- 

vising the detailed information pertinent to com- 
modities or services appearing as an imaged 
object in a picture program, in accordance with 
a request from a viewer connected to the ac- 

25 quisition unit over a communication network; 

and 

a communication functional unit for notifying 
the latest value of the number of times of scene- 
based access, as acquired by said access in- 
30 formation acquisition unit, and as requested by 

a viewer. 

18. A server apparatus comprising: 

35 an access information acquisition unit for ac- 

quiring the information pertinent to the access 
state from a viewer to a database supervising 
the detailed information pertinent to commodi- 
ties or services appearing as an imaged object 

40 in a picture program, in accordance with a re- 

quest from the viewer connected to the acqui- 
sition unit over a communication network; and 
a communication functional unit for notifying 
the latest value of the number of times of com- 

4 5 modity-based or scene-based access, as ac- 

quired by said access information acquisition 
unit, and as requested by the viewer. 

19. A server apparatus comprising: 

50 

an information furnishing unit for accessing a 
database supervising the detailed information 
relevant to commodities or services appearing 
as an imaged object in a picture program, re- 
55 sponsive to a request from a viewer connected 

to the information furnishing unit over a com- 
munication network, for furnishing a list file pro- 
vided for presenting the information relevant to 
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plural scenes in a list form to the viewer, said 
list file being an information file provided for the 
relevant picture program. 

20. The server apparatus according to claim 1 9 wherein s 
said information furnishing unit has the function of 
furnishing an associated Index picture for each 
scene forming the list file. 

21 . A picture information displaying method for display- 10 
ing, on a display picture, the information pertinent 
to commodities or services appearing in each scene 
of a picture program, responsive to a request from 
a viewer, said method comprising: 

15 

a processing step of reading out the results of 
summation of the number of times of scene- 
based access events by the audience to a da- 
tabase supervising the detailed information rel- 
evant to commodities or services appearing in 20 
a scene to be displayed; and 
a processing step of displaying the read-out re- 
sults in association with a scene in question. 

22. A picture information displaying method for display- 25 
ing, responsive to a request from a viewer , the in- 
formation pertinent to commodities or services ap- 
pearing in each scene of a picture program, on a 
display picture, said method comprising: 

30 

a processing step of reading out the results of 
summation of the number of times of commod- 
ity-based or service-based access events by 
the audience to a database supervising the de- 
tailed information relevant to commodities or 35 
services appearing in a scene to be displayed; 
and 

a processing step of displaying the read-out re- 
sults in association with each information rele- 
vant to the commodities or services appearing 40 
in a scene in question. 

23. A picture information displaying method for display- 
ing, responsive to a request from a viewer, the in- 
formation pertinent to commodities or services ap- 45 
pearing in each scene of a picture program, on a 
display picture, said method comprising: 

a processing step of reading out an index pic- 
ture of a scene to be displayed; and so 
a processing step of displaying the index pic- 
ture, as read out, in conjunction with the infor- 
mation relevant to commodities or services ap- 
pearing in a scene in question. 

55 

24. A program information displaying method for dis- 
playing, responsive to the viewer's request, the in- 
formation relevant to commodities or services ap- 



pearing in each scene of a picture program, on a 
display picture, said method comprising: 

a processing step of reading out the information 
relevant to commodities or services appearing 
in each scene, for displaying the information 
relevant to plural scenes in a list form; and 
a processing step of deploying the information, 
as read out, based on the preset layout infor- 
mation, for presentation in a list form on a dis- 
play picture. 

25. The program information displaying method ac- 
cording to claim 24 wherein 

an index picture is displayed in each of said 
plural scenes in association therewith. 

26. A computer-readable recording medium having a 
program recorded thereon, said program being us- 
able for a computer to implement 

a program ancillary data extraction function of 
extracting, from picture program data, program an- 
cillary data relevant to commodities or services ap- 
pearing in each scene; and 

a display control function of displaying the in- 
formation relevant to commodities or services ap- 
pearing in each scene, and the sum total of the 
number of times of scene-based access events by 
the audience to a database supervising the detailed 
information relevant to said commodities or servic- 
es, on a display picture, based on the preset layout 
information. 

27. A computer-readable recording medium having a 
program recorded thereon, said program being us- 
able for a computer to implement 

a program ancillary data extraction function of 
extracting, from picture program data, program an- 
cillary data relevant to commodities or services ap- 
pearing in each scene; and 

a display control function of displaying, on a 
display picture, the information relevant to the com- 
modities or services appearing in each scene, and 
the sum total of the number of times of commodity- 
based or service-based access events by the audi- 
ence to a database supervising the detailed infor- 
mation relevant to said commodities or services, 
based on the preset layout information. 

28. A computer-readable recording medium having a 
program recorded thereon, said program being us- 
able for a computer to implement 

a program ancillary data extraction function of 
extracting, from picture program data, program an- 
cillary data relevant to commodities or services ap- 
pearing in each scene; and 

a display control function of displaying the in- 
formation relevant to the commodities or services 
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appearing in each scene, on a display picture, 
based on the layout information adapted for pre- 
senting the information pertinent to plural scenes in 
a list form to a viewer. 

5 

29. A computer-readable recording medium having a 
program recorded thereon, said program being us- 
able for a computer to implement 

an extraneous data presence/absence detec- 
tion function of extracting, from picture program da- 10 
ta, data indicating whether or not the information rel- 
evant to commodities or services appearing in each 
scene is available through an external database, to 
detect the presence/absence of the extraneous da- 
ta based on the so extracted data; 15 

a downloading function of downloading, when 
an access request to extraneous data is made by 
an operating input by the viewer, a list file provided 
from the outset for presenting the information rele- 
vant to plural scenes to the viewer in a list form, from 20 
an external database, said list file being an informa- 
tion file provided from the outset for a relevant pic- 
ture program; and 

a display control function of opening the 
downloaded list file to display the information rele- 25 
vant to the plural scenes involved in said list file on 
a display picture in a list form, based on the preset 
layout information. 

30. A computer-readable recording medium having a 30 
program recorded thereon, said program being us- 
able for a computer to implement 

a downloading function of having communica- 
tion with another terminal device and of download- 
ing the information relevant to commodities orserv- 35 
ices appearing in each scene from a recording me- 
dium supervised by said other terminal device, said 
information being program ancillary data of a pic- 
ture program in question; and 

a display control function of demonstrating, on *o 
a display picture, the information relevant to com- 
modities or services appearing in each scene, re- 
sponsive to an operating input by an operator. 

31. A computer-readable recording medium having a 45 
program recorded thereon, said program being us- 
able for a computer to implement 

a display control function of displaying an in- 
dex picture of a scene, in an imaged object of which 
are included commodities or services selected with 50 
a will for purchasing, in a correlated fashion, in a 
basket picture in which said commodities or servic- 
es selected can be confirmed prior to final transmis- 
sion of said will for purchasing to a server. 

55 

32. A computer-readable recording medium having re- 
corded thereon a program usable for a computer to 
implement 



an access information acquiring function of 
acquiring, responsive to a request from a viewer 
connected to the computer over a communication 
network, the information pertinent to access states 
from the audience to a database supervising the de- 
tailed information pertinent to commodities or serv- 
ices appearing as an imaged object in a picture pro- 
gram; and 

a communication function for notifying the lat- 
est value of the number of times of scene-based ac- 
cess events as acquired by said access information 
acquiring function and as requested by the viewer. 

33. A computer-readable recording medium having re- 
corded thereon a program usable for a computer to 
implement 

an access information acquiring function of 
acquiring, responsive to a request from a viewer 
connected to the computer over a communication 
network, the information pertinent to access states 
from the audience to a database supervising the de- 
tailed information pertinent to commodities or serv- 
ices appearing as an imaged object in a picture pro- 
gram; and 

a communication function for notifying the lat- 
est value of the number of times of community- 
based or service-based access events as acquired 
by said access information acquiring function and 
as requested by the viewer. 

34. A computer-readable recording medium having a 
program recorded thereon, said program being us- 
able for a computer to implement 

an information furnishing function of access- 
ing a database supervising the detailed information 
pertinent to commodities or services appearing as 
an imaged object in a picture program, responsive 
to a request from a viewer connected to said com- 
puter over a communication network, for presenting 
a list file provided from the outset for presenting the 
information relevant to plural scenes to the viewer 
in a list form, said list file being an information file 
provided from the outset for a relevant picture pro- 
gram. 




EP 1335 592 A1 136 



5 



10 



15 



20 



50 



69 



EP 1 335 592 A1 




70 



EP 1 335 592 A1 



CO — 




UJOquJ 
J"=>OC< 




=> s— 
o 
o 




zz ^ 

K SEE- So 

mj<520ul 
:oocq«<q 




DC 
< 



< < < < 
*~ *Z I- C 

< S < < 

2s u o o 0 

F=UJ S Z UJ 2 

<H- uj n ^ 

jyco x S o t= 

cc>- o 2 P o 

OC/> CO — C/> LU 



Q 



2o 



Ouj 

0(0 



ZlU 
OS 
Z Zh 

>:tr cccqo 

s:o os< 
8i siS 




CM 
(3 



20.0 



cco£-§ 
io<<cc 

UJCEZO 
Ofc-<0 

<0^a. 
o 



71 



EP 1 335 592 A1 



SYSTEM 
CONTROLLER 


EBB 




of 





-a 
o 




§3>« 

GO 



Q 

cc 

CD 



OCj- 

Illl 



0- 




o 



METAMOR 
GENERATING 
COMPUTER 




MATA POCKET 
GENERATING 
COMPUTER 


□i 



8. 



DC CO 5 Ui 



II1IO 



CO 



8 



RETRIEVING 
COMPUTER 


, 1 


WWW SITE 
CREATION COMPUTER 


of 



2S 



a. 

§55 
8 

CC 



CO 



CO 



72 



EP 1 335 592 A1 



1- 



CD 



a. 
O 



3 

CM 

jL 



5!< 




73 



EP 1 335 592 A1 



HI 






ay 






< / 


So F 




GO / 






gill 




|( 


















oc 






Ui 






CO 






2 






LU 






s 




/ 




74 




75 



EP 1 335 592 A1 



26 



1 



SPONSORSHIP REQUEST CONDITION DATA 



MANAGEMENT ID 



INFORMATION ON REQUEST SOURCE 
(URL, e-mail ETC) 



DESTINATION OF TRANSMISSION 



NAME OF REQUESTER 



PROJECT NAME 



DATE AND TIME FOR EXECUTION 



PROJECT CONTENTS 



TARGET LAYER 



DESIRED CONDITION FOR SPONSORSHIP 



/-26A 

r26B 

^26C 

^26D 

<-26E 

-26F 

-26G 

-26H 
-261 



FIG.7 



76 



EP 1 335 592 A1 



( START ) 



READ OUT INPUT FORM 



S1 



S2 



DISPLAY INPUT FORM ON SCREEN 




DECIDE ON DESTINATION DISTRIBUTION 



S5 



CREATE TRANSMISSION DATA 
WITH MANAGEMENT ID 



.z: 



S6 



SAVE AND SEND TRANSMISSION DATA 



( END ) 



FIG.8 



77 



EP 1 335 592 A1 



DISPLAY 
MEANS 



'20J 



20H 



OPERATING 
INPUT 



20F 



± 



20A- 



DISPLAY CONTROLLER 



DISPLAY 
FORM 
RECORDER 



r 

30A 



CONTROLLER 



REPLY DATA 
RECORDER 



30B 



201 

A 



COMMUNICATION 
UNIT 



REPLY RESULT RECEIVING DEVICE 
(DEVICE FOR INPUTTING CONDITIONS 
FOR SPONSORSHIP REQUEST) 



20G 



30 



NETWORK (PUBLIC AND IP NETWORK) J 



22- 



REGISTERED 
MEMBER 
TERMINAL 



FIG.9 



78 




79 




EP 1 335 592 A1 



32 
\ 


REPLY RESULT DATA 


MANAGEMENT ID (RESPONDER) 


^■32A 


MANAGEMENT ID (REQUESTER) 


^32B 


INFORMATION ON REPLY SOURCE 
(URL, e-mail ETC) 


^32C 


DESTINATION OF TRANSMISSION 


^-32D 


NAME OF REGISTERED MEMBER 


^32E 


CONTENTS OF COMMODITIES, SERVICES THAT 
CAN BE OFFERED UNDER SPONSORSHIP 


^32F 


CONDITIONS FOR SPONSORSHIP 


^32G 











FIG.11 



80 



EP 1 335 592 A1 




CM 

d 



81 



EP 1 335 592 A1 



E 



O 

o 
co~ 



o 

UJ 



o 
o 
2 

22 



g 



co 



o 

CM 



o 

LU 
O 
> 
DC 
UI 
CO 
LL 

o 

Z ; 

g 

CO 

> 
5 



cc 

cm 



a 

UJ 



< 
x 

CO 
UJ 

o 
> 

Ui 
CO 
LL 
O 
Ui 

< 



CO 
CM 



z 

UJ 
UJ 

3 



CD 

CM 



< 

CC 

co 
5 

UJ 

cc 

LL 
O 
UJ 

< 



o 

CM 



UJ 

Q 
Q. 
3 
LL 
O 
UJ 



CC 
UJ 

cc 

UJ 

co 
o 

UJ 

cc 
u. 

o 

UI 



ULI 

cc 
o 
*c 

O | CO. 
I— 

£9 
cc 
I— 

CO 

Q • ac 

CO 

£ I o 

| CO 

CO 

;r o 



O 
UI 

cc 

CO | g 

o 
E 



< 

CM 



o 



CM 



CC 

o 

CO 

CO 
CO 
UJ 

z 

CO 
3 
CO 



Q 

CM 



UJ 

CM 



LL 

T— 

CM 



CO 
CO 
UI 

z 

CO 
3 
CD 
Li. 

o 

UJ 

t 

CO 



CC I CO 
UJ LU 
CD | CC 
, Q 
3 Q 

2 < 
UJ , 

b. s 

UJ w 

O UJ 

z I o 

UI . 
' UJ 
Q 

z 
o 
a. 

CO 

ui 
oc 
cc 
o 
o 



o 
a. 

CO 
UJ 

cc 
oc 
o 
o 



cc 

3 
UJ 

o 
z 

UJ 

o 
z 
o 

Q. 
CO 
UI 

cc 
cc 
o 
a 



CO 



CM j 04 I CM 



oi £ 

CO 

i s 

cc 1 
o 
u. 

5j < 

O UJ 

pig 

Q 

Q | Z 

a X 

o o 



a 

UJ 



o 
> 

oc 

UJ . — 

co I < 

CO , 
UJ CO 

E ty 
o 

5 

o 
a 
u. 
o 
z 
o 

£2 
> 



Q 
UJ 

- . -J 
Z O 



CO 

(3 

Urn 



CM I CM I CM 



CM 



UJ 

co I S 

B 2 

O I DC 

a. 

o 
ui 



O I tt. 

T— J T— 

CM CM 



82 



EP 1 335 592 A1 



CO 



CM 
r-. 

CO 



N 


P REQUEST CONDITION 






X 






<n 






OC 






O 






</> 






z 






o 






Q- 






<r> 






ID 






cc 












1- 






O. 






< 












Eoiuo 

SiSi 

OsujO 

Z£j eel- 
's UlUJ 



83 



EP 1 335 592 A1 



DISPLAY 
MEANS 



-22H 



22F 



OPERATING 
INPUT 



22D 



1 



22A- 



DISPLAY CONTROLLER 



CONTROLLER 



REQUEST CONTENT 

CONFIRMATION 
FORM RECORDING 
UNR" 



22B 



SPONSORSHIP 
REQUEST DATA 
RECORDING UNIT 



~~r~ 

22C 



COMMUNICATION 
UNIT 



SPONSORSHIP REQUESTER RECEPTION DEVICE 
(REGISTERED MEMBER TERMINAL) 



22E 



22G 



22 NETWORK (PUBLIC AND IP NETWORK) 



20- 



SPONSORSHIP 
REQUESTER 
TERMINAL 



FIG.15 



84 




85 



EP 1 335 592 A1 



DISPLAY 
MEANS 



-39 J 



39D 



OPERATING 
INPUT 




39A— DISPLAY CONTROLLER 



GENERIC TERM 
REGISTRATION 
FORM RECORDER 



DESIRED 
DISTRIBUTION 
CONDITION 
RECORDER 



39B 



SPONSORSHIP 
CONDITION 
PRE-REGISTRATION 
FORM RECORDER 

\ 

39C 



INPUT DATA 
RECORDING 
UNIT 



39E 



f^?JSl?J5AU^ ,MFORMATfON ,N PUT DEVICE 
(SPONSORSHIP REQUESTER RECEPTION DEVICE) 



39 



NETWORK (PUBLIC AND IP NETWORK) 



35- 



SPONSORSHIP 
TRANSACTION 
BROKER SERVER 



FIG.17 



86 



EP 1 335 592 A1 



5*. 




cc 
O 



< 
O 

LU 

CC 

s 

LU 
t 

o 

CC 
LU 

z 

LU 

o 



GO 

g 

LL 




UJ 



c/> 



LU 

S 



O LU 

W Q LU C J 

UJ So? o 

1= Olu cc 

CO OCO Q. 



O Q Uj U. CD 

O O O O o 
\fr ^ \f- ^ 



si 



87 




88 



EP 1 335 592 A1 



< 

Q 

LU 

o 
< 



< 
o 

o 

Q 

ID 



1 < 

4 



o 



• <Ui 
DC CO 

OC 
CL 



< 

< 

Hi 

E 
a 

In 

O d 

oz 

OLu, 
Q<Z 

— h- LU 



z 
o 

5 

2 

z 

CO 
CL 

o 



111 

>- u. tZ 

5t cess 

O LU o 
OflLCL 



8 

I I I I I 



2 



UJ 



2. b 



LU 



uT 

— Q 
UJ O 

§uT" 

f8i 

Ol-DCZ 

S88SS 

<SSZZ -i 
OCOOOOOC 
CDOOOOO 



CO 
UJ 



o 

LU 

m 

3 



oc z 
Oco 

*= CO 
= ZlU 

guig 



>- 
oc 

UJ 

> 

UJ 
Q 



oc 
o 



LU ^ 

oc2co 

Q_ O 



1 1 I i I I I I I 1 1 



go O 
oco 



o 

s 
oc 
o 



8^g 



3 5 



UJ 
C5 
< 

Q 



OQ2 

SFS G 

CL<«g 



CM 

d 

LL 



5 



< ^ 
OC CO 

uj°o 

< ^ 



89 



EP 1 335 592 A1 



CO 
CC 
Ui 
S 
oc 

>2 
1 55 

E °- fn 
> o ^ 

CC HI | 
O rj 2 

q 2 s 

tz 2 ui 

5 UJ 2 

< CC fc 
£T < Z 

O a cc 
a. 



CO 
LLf 

CC 
</J 

s 

X 

uj i 5 
m EC 

Ui 2 JS 
-J ft Ui 

b < * 

CC - o 

SIS 
5 oc S2 

2 > °t 

~* CD O 

i 9 £ 

i = s 

J J ? 

I I I 



CO 
UJ 

o 

X 
CO 
CC 

ui H 
^ < 

Ui 

> 
o 



o 



a 

oc 
^ < 
s o 

< Q § O 

- cc 

2 



LU 

2 > 



a 

< LL 



O 
Q 



CO 
CC 



CO ^ ^ 



2 2 £ £ 

£ < 3 a * 



I I I I I 22. 



°- CO O § 

h c o o 2 

cc o a a o 

o a. o o o 

aw j jo 



CO 
QC 

£ co 



UJ 

loo 

s z < 

Q O DC 

Z (O H 

< UJ □ 

o S 5 — 

■r UJ 3 2 

2> x o o 

g H- CO 00 



CC 

o 



CC 3 

cog 
CC — 
UI o 

5 55 



a. 2 2 



So 
o <5 S 
z < 2 
E^s 
o-{o 

UJ d Z 

Q £ < 

Z 5? h 

< o o 

7 ZUJ 
? < u. 

2 > < 
cos 
o s g 

got 
C uj a 

UJ u I 
CO UJ O 
Z> —I UJ 

- £ » 

< o < 
a. «= 



CM 

O 
LL 



I l l I I l I I 



OQ 



90 



EP 1 335 592 A1 




CM 
CM 

CD 

LL 



91 



EP 1 335 592 A1 



h 
ii 

Q. 

o 



z 



8>2 




CO 
CM 

CD 

LL 



92 



EP 1 335 592 A1 



o 

1*5 

a. 
O 



8 



a 




CM 

d 

LI- 



'S 



93 



EP 1 335 592 A1 




LO 
CM 

d 

Li. 



5 JS * 



94 



EP 1 335 592 A1 



Uj k CD 

tj. ^ \r- 
to 



8 



8 




CO 
CM 

O 
U- 



95 



EP 1 335 592 A1 



6 
o 



in 5£ 





CO 



So 

8s 



3 



si 

CO 



ooo 
o o o 

~ 2 ° 
o o o □ 
o o o □ 

O O O X 

Q O O X 

O O O X 

O O O X 

O O O X 




-CO 



CM 

d 

-S8 £E 



in 



.CO 

to 



96 



EP 1 335 592 A1 



c 



START 



J 



S2/ 



READOUT INPUT FORM 



I 



DISPLAY INPUT FORM ON SCREEN 



< 



S23 



IS THERE SCENE SELECTION OPERATION ? 



| YES 



> 



NO 



J2 



S24 



RECORD CODE INFORMATION OF SELECTED SCENE 
AND DISPLAY IT WITH RELATION TO INPUT FORM 



< 



IS SPECIFIED INPUT ITEM SELECTED INPUT ? 



rS25 
^ NO 



> 



YES 



S26 



LIST INFORMATION PROVIDED 
FOR RELEVANT INPUT ITEM COLUMN 



S27 



DISPLAY SELECTED INFORMATION 
IN INPUT COLUMN 



S28 



DISPLAY INFORMATION 
INPUT BY OPERATOR 
IN INPUT ITEM COLUMN 



S29 



\RELEVANT TO SPONSORSHIP COMMODITIES ? / 




YES sS30 


ENTER CHECK MARK IN CHECK ITEM COLUMN | 









c 



END 



FIG .28 



97 



EP 1 335 592 A1 



c 



START 

~T~ 



S31 



READOUT INPUT FORM 



S32 



DISPLAY INPUT FORM ON SCREEN 



< 



rS3 3 



IS THERE SCENE SELECTION OPERATION ? 



> 



NO 



YES 



r 



S34 



DISPLAY SELECTED INFORMATION IN INPUT 
ITEM COLUMN OF DRAG & DROP DESTINATION 



< 



IS INPUT INFORMATION 

RELEVANT TO SPONSORSHIP COMMODITIES 



S29 



NO 



YES 



S30 



ENTER CHECK MARK IN CHECK ITEM COLUMN 



C 



END 



FIG.29 



98 



EP 1 335 592 A1 




99 



EP 1 335 592 A1 



t 



68A 



INTERFACE 



68C 
\ 



73 



IMAGE PROGRAM 
DATA CREATING DEVICE 



COMMUNICATION 
UNIT 



68F 



PACKET 
PROCESSOR 



68B 



69 



ACCESS 
INFORMATION 
ACQUISITION 
UNIT 

r~ 

68D 



ACCESSING 
NUMBER OF TIMES 

ACCORDING 
UNIT 

5 

68E 



FIG.31 



100 



EP 1 335 592 A1 




CM 
CO 

C3 



101 



EP 1 335 592 A1 



2 "J 
?! ► 

CM 
CO 



C\J CO ^ 

P P P P 

fc fc fc 

■ h ( , t 



% 

LU 
< 

< 

o 
cc 
a. 

a: 
< 
m 

LU 



EC 

o < 

o ► 

o 

P Q 



< 
o 



i 



I 



CO 

=J 



g 

o 

SCO 
SUJ 

o9 
o> 

^85 

2 



8 



A-1 


? 




< 


I 



r- 
I 

CD 


CM 

I 


CO 


DO 


I 

CD 



Of I 




7 


I 


o 



£ x « □ 



8* 



<i □ 



-Mm 



S 

x x 
O 



< 



X O « □ 

9 ^ : ^ 



ci ui ll 



102 



EP 1 335 592 A1 



8 



< 
ec 
o 
o 
cc 
a. 

cc 
< 

CD 
LU 



SI 



a 

O LU 

?! ► 

CM 





oc 


o 
o 


STA 


o 


► 


o 




v at 


< 

DC 








1 



_ 3 



s 



O 
Q 



s 



! 



o 



£§ 

o 

a. 
co 



T 
< 




CO 


1 

< 


1 

< 



— }- 

— a: 

_ ► 



8 X 



□ 
□ 



T- N O 



2 

O 

2 x 

x x 

o ■ 



o 

LU 

Q 



O 
O 



co 



a) 
3 



1 



Q UJ U. 



• 



103 



EP 1 335 592 A1 




104 



EP 1 335 592 A1 



LU 

< 
cc 
o 
o 
or 

CL 

< 

00 
IU 



O ill 

* ► 

00 



o 
O 

Sco 

o£> 
o> 

Xco 

z 
o 

, CL 

i to 



I 




T 




I 


< 





8 X 



□ 



2 

ix 

X x 

CD 




1 

S 

CL CL 



UJCO 



~~ o 



8 



CL 

cv 

*T 6 

0 u 
o 

- i 

^ E 

1 I 



O 
O 

o 



5 
S 
5 



O 
Q 



o 
o 



5 
5 



O 
CP 

a 



CO 
CO 




UIt- 
O O 



LU LO 
W2 



o 

SO 

x O 

9 ^ »o 
a 



4 □ 



□ LU U- 



«_-. .. . . . j 



105 




106 



EP 1 335 592 A1 



( START ) 

I rS4 1 



DISPLAY COMMODITY SERVICE INFORMATION 
IN INPUT ITEM DISPLAY AREA AND DISPLAY 
INDEX IMAGE IN IMAGE COLUMN OF INPUT 
INFORMATION DISPLAY AREA 



IS INFORMATION OF INPUT ITEM DISPLAY 
AREA SELECTED ? 




YES 



S43 



DISPLAY SELECTED INFORMATION 
IN RELATION TO INDEX IMAGE OF 
DESTINATION OF DRAG/DROP 



C 



END 



FIG.38 



107 



EP 1 335 592 A1 



( START ) 



DISPLAY COMMODITY SERVICE 

INFORMATION IN INPUTTTEM D^PLAY AREA 
DISPLAY INDEX IMAGE IN IMAGE COLUMN 
?£JNSyi ^FORMATION DISPLAY COLUMN 
WITH EACH SCENE IN DATA COLUMN 



S46 



IS INFORMATION g^NPUT HEM DISPLAY \ NO 



I YES ^47 



D,SP VA Y SELECTED INFORMATION IN DATA 
COLUMN OF INPUT INFORMATION 
DISPLAY AREA AS DRAG/DROP DESTINATION 



r I 

C END 



FIG.39 



108 



EP 1 335 592 A1 



— i 



I- 

OO 



a 
o 

8 s 

CL 
UJ 



JO' 
00 



li 

a. 
O 



i 

CO 



o.< 



n ^ 



2>s£2 
owgo 

O z 



OO 



z 
o 

< 



3D 

s 
s 

8 



00 



CD 
oo 



oo 
op 
<o 

<£E 

OUJ 

°<r 



oo 



o 

oo 



CD"- OC 
ZO UJ 

asps 
o=> o 
<z o 



00 
oo 



oc 
o 

CO 
CO 
LU 

o 
o 
oc 

0L 



< 
o 
o 
o 
< 

< 
oc 
a 
o 
oc 



JO 

oo 



109 



EP 1 335 592 A1 




110 



EP 1 335 592 A1 



o 
z 
o 

Qz 



CO 



oo 



Q_ 

o 



O 

JO 




I 

00 



cc 

o 



O 

z 
< 
o 
=J 

CO 



cc 
o 



00 



0"r tC 

ZO uj 

03 O 

<z o 



oo 



cc 
o 

CO J 

</> t 



Si 

Q ! 
Q 
Q 
< 



< ■ 

cc 

8 

CC 

0L 



oo 



CM 

d 



111 



EP 1 335 592 A1 




112 



EP 1 335 592 A1 



DISPLAY 
MEANS 



87A 



-87H 



DISPLAY 
CONTROLLER 



PROGRAM 
ADD DATA 
RECORDER 



V 

87B 



87D 

\ 



87G 



OPERATING 
INPUT 



CONTROLLER 



INPUT DATA 
RECORDING 
UNIT 



87C 



COMMUNICATION 

UNIT 
(DOWNLOADING) 



87E 



RADIO 



TERMINAL 
(SUCH AS TV 
RECEIVER) 



85 



x J 



PROGRAM ADD DATA PROCESSOR 
87 



FIG.44 



113 




114 



EP 1 335 592 A1 




CO 

u. 



115 



EP 1 335 592 A1 



8- 



o 
o 

■ 

M 
N 



CL 


CL 


CL 

• ■ ». 


Q. 


a 


• 

O 


■ 

o 


CO 
vv 


• 

o 

/ % 




o 

>* 
>> 


o 

• 

X 


3s 
5 


UU. ( 


tt. c< 



o 
o 



UJ 

S| 

en pp. 



5 



5 

5 



>- 
>■ 

M >" _ 

M >" X 

M >~ X 

Nl >- X 

N >- X 



5 

5 



5 



5 



3 
5 



os 



3 

2 i- co 

? h w 

3 I- W 
3 H- CO 



2 



S3' 



LU 



O 

O 

s 

o 
o 



< 


CD 


O 


< 


to 


O 


< 


CO 


O 


< 


CD 


O 


< 


CD 


O 


< 


CD 


O 



jo- 



CO 

SJ 
6 
z 

III 

z 
111 
o 

CO 

o 
o 

CL 

s 



o 
o 

s 
o 
o 



LU 

S -j 

QC LU 

O QC 

u- < 

DC Q. 

UU Q. 

CL < 



O 
Q 
Q 
Q 
Q 
O 



co 

LU 

UJ CO 



UJ u. 

Ui LL. 

LU UL 

UI UL 

LU LU 

Ui LL. 



< 

s 

cc 
< 

X 



CO 

< 

a 
z 

3 
CO 



Ui 

-J 

CD 



< 

LL. 

o 

CO 



a 
o 
o 
a 
a 
a 



a. 

=> 
o 



o 

I 

o 



o o o o o o o 



OS 



I 

os 



S3 

OS 

VI 



52 

OS 



111 

p 



OflC 



g 

LL 



116 



EP 1 335 592 A1 



8- 



cfc .& S: ol £: 



i 



o 
o 

N 
IM 

3 
5 



o 
o 

>» 
3 



o 
o 

X 
X 

3 
3 



o 

o 

35 

S: 
3 



o 

o 



3 

3 



8| 

o 



>■ 
>• 

m >- >< 
M >- x 

M >- X 



LU 

< 



CO 

UJ P 

z o 
UJ o 

q s 
t/> s 

V— o 

o 
o 
a. 

< 



UJ 



uu 



3 
O 

5 



UJ 



o 
oc 
o 



< 

UJ 

5 



er: « 

O O 



s 

oc 
O 

LU 
Z 

r> 



55- 



o 

o 



9 32 



3 



5 s 2 E 

* 5 t 

s 2 

S 3 H- 

S r> i- 



o 
o 

■ 

3 



CO 
CO 
CO 
CO 





CD 


O 


o 


UJ 


LU- 


O 


< 


00 


o 


Q 


UJ 


LL. 


a 


< 


CO 


o 


O 


UJ 


LU- 


a 


< 


CD 


o 


Q 


UJ 


LL. 


<s> 


< 


CO 


o 


Q 


UJ 


U, 




< 


CD 


o 


Q 


UJ 


UU 





LU 

o 
< 

oc 

LU 
> 
LU 
CD 

X 
X 



o o 



UJ 

cc 
< 

O 



t 

2 



1 



>• 

=> 
(0 



GO 

g 

LL 



m 



CD 

S3 



oc 
oo 



I 



117 



EP 1 335 592 A1 




O) 

d 

UL 



118 



EP 1 335 592 A1 



Is. 

"o" 



cm 

CM 

6 
Z 
Ui 

z 

Ui 

o 

GO 

o 
o 

CL 




o 
o 

5 



o • o 

u o o 

o . 

3 • V) 

3 C «f» 



* £ 5 5 

5 £ £ £ 

5 5 3 5 



Bit 

| i t S 

^ 3 h W 



8 





CD 


O 


< 


CO 


O 


< 


CD 


O 


< 


CO 


O 


< 


CD 




< 


CD 


O 



o ui 

o ui 

o Hi 

O UI 



Ui 

5 
cc 

2 

CC 
Ui 
CL 

O. 



cc 
< 

0. 
CL 

< 

o 



cc 
< 



£3 



o 

o 
o 
o 
o 



9= 

o 



to 

CO 

O j < P 



□ □ Q 

a a a 

Q □ □ 

Q □ Q 



g- 



CO, 




S 



I 









-J 






-J 






< 






AM 




BUY 


Lu 




s 













CD 
S3 



ui 

el 

QO 

op 



occ 
oo 



in 



119 



EP 1 335 592 A1 




120 




121 



EP 1 335 592 A1 




122 



EP 1 335 592 A1 



a. 



< 



O 

00 

ZD 

o 



O 
O 
a 
x 

CO 

<o 
=> 
o 

LU 

a 

o 
o 

too 

E9 

acc 

ii 

ii 

u-O 
Og 
uj< 

<cc 



& co 
CO 

cc < 

Si- 
ll 

§8 

OUJ cc cc 
O.CL < =d 

too 3: o 



to £ 

DC ^UJ 

jy £c2 

2 CD < 

CO m 

U- EL <5 

o tt § 

cc — 
S a. t 
copra 
s to z 
=3 o o 
zoo 




98K 


<< 


A 






V 
V 


3: ^~ 





-J 



2 



00 



28 



ccco 

Occ 

li 

o< 

a. cc 

a. 
O 



D D a 



lO 

d 

UL 



CO 



CD 



< 
s 



<^ re 



123 



EP 1 335 592 A1 



O 
CD 



Q SI 



Q 
UJ 
CC 

< 

C3 
z 

UJ 
£0 

S 
< 

DC 
C5 
O 
DC 
CL 



E 



< 
> 

UJ 

—I 

UJ 

tr 



g 

-J< 

see 

<fO 



UJ. 



<Ui 

ccS. 



a 

o 

Q 
O Q 

o a 

COO Q 

fl) O Q nj 

< CO O Q uj 

< CD O O uj 

< CD O O LU u- 

< CQ O Q UJ t 

< DQ O Q UJ u. 



CO <2 CO 

< < < 
o o o 

Q 9 Q 

< s < 

o o o 

CQ CP ED 

CO O O 
CO CD 00 
-3 < 



r— rr vr 

222 

o o o 

CC CC DC 
CO CQ CO 

CO co eg 



o o o o <_> o 

I** CO CO CO CO CO 



a 

UJ 

oc 
< 
z 

LU 
UJ 
CD 

o 



< LU 

cc z 

§1 

DC O 

a. 



o 
a 



333 

888 

ooo 
cccccc 

CO CQ CD 
CQ 00 CO 

*r *r 



CO CO CO CO 

< < < < 
o o o o 

Q Q Q O 

< <t < 
OOOO 
CC CC DC CC 
CO CO CO CQ 
CQ 0Q 00 CQ 

~» -3 -3 -> 



O O O O O O O 



UXONOCOOt- 



B» I I B 



z 
< 
> 

UJ 



Q 
UJ Hi 



CC 
< 



z 
g 



oc 

Q 



T— T- T— T- T— 

JL ' JL * ' ' ' 

o t- o cm co <o 



LO 

m 

m 

(D 
u_ 



BE 



*r_i ° 
f— -± CC 



LU H 
OCUI 



t lO ID N CO O) O 

1~ T- T- T- T- T- CM 

X) -Q -Q -Q -Q X> -Q 

i£i£i£i2i£i£i£ 



124 




125 



BP 1 335 592 A1 



CO 



Q 

LU 
>• 

3 

a. 

CO 

a 

UJ 

m 
O 

UJ 
Z 
LU 
O 

CO 



o 

LU 
CL 

CO 



CM 

to 
CO 



li 

<<0 

oo 

Z — 

>< 
?* 

go 
SCO 

Sz 
o- 

P 

o< 
quj 
<o- 

LUCL 
0C< 



CO 

*o 

CO 

M 



CO 
LU 

Slu 
F2 

SB 

Sib 

DO 

oft 
2z 

CO— 
(OZ 

Sg 
q5 



LU 



s 

CO 



UJ 

O 
< 

s 

O 



3 

•3 

S 
CC 

2 



CO 

M 



32<o 

O.C0CC 

co<5 

qGDu. 

zoo 
>-<< 

-ICCOC 

Q-OO 
LULLLU 



CO 



CC 
CO 



z 
o 

i» 

OS 

£o 
5 co 

LL-J 

o< 

S23 



>-LU 

CL 

CO 

a 



50 

CO 

M 

CO 
LU 



o 

LU 
>- 



CL 
CO 

O 
LU 
CO 

o 
co 

LU 

z 

LU 

o 

CO 
—J 

< 

CC 
3 



o 

LU 

a. 

CO 



jo 

CO 

M 



z 
2 ° 

so 
SS 

U.LU 
20 

yS 

>LL 

CCo 
LU 

CO "J 

tS 

to 

QCO 

§x 
|o 
s< 

OLU 

o z 

55 

oz 
OS 

CCq. 
a. 
< 



io 

CO 

M 



CO 

co£ 

UiZ 

Sgiu 

oca: 
So 
|g 

i?J 

o< 
zee 

553 

oz 



o< 

<cc 

LUO 



io 

CO 

M 



o 

CO 
CO 

SI 



^co 

0°LU 

§58 

LU — I 

a:z< 

CLCO 0 " 
CO<LT 
qCQO 

i§§ 

osS 

o-o£ 
lullO 
QZ<L 



IO 

S3 



o 

z 

LU 



126 



EP 1 335 592 A1 




00 

m 

LL 



127 



# 



EP 1 335 592 A1 




I- 



O 
GC 
f— 
Z 

o 
u 



I- 



li 



LU 
O 

> 
LU 
Q 

CC 
LU 
> 

in ! cc 

CS - 1 Ul 

"I CO 



cogg 
2< 



I 



2t 

3 Il 




in 

d 



cogl=> 

OZoO 

< o 



128 



EP 1 335 592 A1 



CO 
CO 

LU 

o 
z 

UJ . 
Q 



< 

s 

5 
O 

o 



CO 

CO 
SI 



Q. 

CO 

Q 

I- 
LU 

O 

o 

Q. 
< 



LU 

o 



o 
< 



cr 



CO 
CO 
CO 



z 
o 

§ 

a 



o 
o 

a. 
< 



jLJ- 



E8 

qQ 

eg 

"cc 
o 



z 
o 

< 
cr 



< 

< 

Si 

-J 

LU 



Qai 
a: 

CO 



CO 



CO 



CO 



111 

a 
a. 

DC 
LU 



3 
O 

o 

CO 

c/> 

LU 

a 
o 
< 



CO 



< 

o 

z 
a 
z 
o 

a. 

CO 
LU 
OC 
DC 
O 

o 

I— 

s 

DC 
I- 

co 
o 



CO 

VI 



CO 

SI 



CO 



CO 



CO 

SI 



a. 

CO 
Q 
O 

z 

LU 



UJ 

a 
o 
< 



CO 

a 

ll 



CO 



a 
z 

CO 
CO 
LU 
O 

o 

OC 

a. 

i— 
a. 

UJ 

O 

a 
< 
oc 

LU 

a 

oc 
o 



129 



EP 1 335 592 A1 



CO K 00 
CO 00 00 

CO CO CO 




r 1 ! 



CD 
Li. 



DC 
Ui 

Ziii 

aw 

cc 

CO 



US 

a. co 

o£ 

CC 
CQ 



CD 



130 



EP 1 335 592 A1 



INTERNATIONAL SEARCH REPORT 



International application No. 

PCT/JP01/09881 



A. CLASSIFICATION OF SUBJECT MATTER 

Int. CI 7 H04N5/445, 7/08, 7/173, G06F17/60, H04H1/00 



According to International Patent Classification (IPC) or to both national classification and IPC 



B. FIELDS SEARCHED 



Minimum, documentation searched (classification system followed by classification symbols) 

Int. CI 7 H04N5/44-5/445, 7/08-7/088, 7/16-7/173, G06F17/30-17/60, H04H1/00, 
H04B1/06-1/16 



Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 
Jitsuyo Shinan Koho 1922-1996 Toroku Jitsuyo Shinan Koho 1994-2001 

Kakai Jitsuyo Shinan Koho 1971-2001 Jitsuyo Shinan Toroku Koho 1996-2001 



Electronic data base consulted during the international search (name of data base and, where practicable, search terms used) 



C DOCUMENTS CONSIDERED TO BE RELEVANT 



Category* 



Citation of document, with indication, where appropriate, of the relevant passages 



Relevant to claim No. 



JP 11-7453 A (Mitsubishi Electric Corporation) , 
12 January, 1999 (12.01.1999), 
Full text (Family; none) 

JP 11-69249 A (Toshiba Corporation) , 
09 March, 1999 (09.03.1999), 
Par. No. [0046] (Family: none) 



JP 2000-69383 A (Matsushita Electric Ind. 
03 March, 2000 (03.03.2000), 
Full text (Family: none) 

JP 11-225294 A (PFTJ, Ltd.), 
17 August, 1999 (17.08.1999), 
Full text (Family: none) 

JP 2000-350976 A (Toshiba Corporation), 
09 March, 1999 (09.03.1999), 
Par. No. f0046J (Family: none) 



Co., Ltd.), 



1-4, 17-22, 
26-27,32-34 



1-4,17-22, 
26-27,32-34 



5-15,23-25, 
28-31 



5-15,23-25, 
28-31 



16,31 



Further documents are listed in the continuation of Box C. Q See patent family annex. 



* Special categories of cited documents: 

"A" document defining the general state of the art which is not 

considered to be of particular relevance 
"E* earlier document but published on or after the international filing 

date 

"L* document which may throw doubts on priority claim(i) or which is 
cited to establish the publication date of another citation or other 
special reason (as specified) 

"CP docu m ent referring to an oral disclosure, use, exhibition or other 



*P" document published prior to the international filing date but later 
than the priority date claimed . 



T* later document published ofter tba international filing date or 
priority date and not fn conflict with the application but cited to 
understand the principle or theory underlying the invention 

*X" document of particular relevance; the claimed hiveatian cannot be 
considered novel or cannot be considered to involve an inventive 
step when the document is taken alone 

document of ppti culfly rcltvflnccj the cfaniicd iDYCjifjon cmnoi be 
considered to involve an inventive step when the document is 
combined with one or more other such documents, such 
combination being obvious to a person skilled in the art 
document member of the same patent family 



Date of the actual completion of the international search 
28 January, 2002 (28.01.02) 



Date of mailing of the international search report 

05 February, 2002 (05.02.02) 



Name and mailing address of the ISA/ 

Japanese Patent Office 

Facsimile No. 



Authorized officer 
Telephone No. 



Form PCT/ISA/210 (second sheet) (July 1992) 



131 



EP 1 335 592 A1 



INTERNATIONAL SEARCH REPORT 



International application No. 

PCT/JP01/09881 



C (Continuation). DOCUMENTS CONSIDERED TO BE RELEVANT 



Category* Citation of document, with indication, where appropriate, of the relevant passages Relevant to claim No. 



JP 10-105599 A (Hitachi, Ltd.), 
24 April, 1998 (24.04.1998), 
Pull text 
& US 5905973 A 



16,31 



Form PCT/ISA/210 (continuation of second sheet) (July 1992) 



132 



